How to use multiple physics engines

Requests regarding how to set up experiments in ARGoS.
jharwell
Posts: 67
Joined: Thu Jan 25, 2018 4:12 am

How to use multiple physics engines

Postby jharwell » Tue Sep 25, 2018 9:27 pm

Hi,

I'm trying to use multiple physics engines to divvy up the processing of robots distributed throughout the arena, so I can run with a 1024 robot swarm of homogeneous controllers, but can't find any examples of how to do so. There is a single reference to a section <arena_physics> in the input file that is for this purpose in the documentation that I could find, but no additional information of about its format/how to use it.

Is this ability still part of argos? If so, is there an example of how to use it I'm not finding?

Thanks!

pincy
Site Admin
Posts: 632
Joined: Thu Mar 08, 2012 8:04 pm
Location: Boston, MA
Contact:

Re: How to use multiple physics engines

Postby pincy » Sat Sep 29, 2018 11:35 pm

I'll make an example available in the next days. I am at IROS, so I'll be a little slow, but I'll do it.
I made ARGoS.

jharwell
Posts: 67
Joined: Thu Jan 25, 2018 4:12 am

Re: How to use multiple physics engines

Postby jharwell » Sat Oct 06, 2018 2:10 am

Awesome, thank you!

jharwell
Posts: 67
Joined: Thu Jan 25, 2018 4:12 am

Re: How to use multiple physics engines

Postby jharwell » Wed Oct 17, 2018 5:24 pm

Hope IROS went well! Any update on the mult-physics engine example?

- John

pincy
Site Admin
Posts: 632
Joined: Thu Mar 08, 2012 8:04 pm
Location: Boston, MA
Contact:

Re: How to use multiple physics engines

Postby pincy » Sun Nov 18, 2018 9:56 pm

It's in the ARGoS examples!
I made ARGoS.

jharwell
Posts: 67
Joined: Thu Jan 25, 2018 4:12 am

Re: How to use multiple physics engines

Postby jharwell » Tue Dec 04, 2018 6:41 pm

I've been playing around with the # engines/ # threads parameters in the multiple_engines example, and haven't been able to obtain real-time results for anything over 200 robots or so. I've modeled my configuration after what was described in the original ARGoS paper: 16 cores/16 physics engines/16 ARGoS threads with 32 GB memory. I'm using a very simple controller which performs a correlated random walk as it searches for objects to pick up (originally built from the ARGoS foraging example), which should be similar to what was used in the paper in terms of computational complexity.

As I'm trying to remedy this, I have a few questions:

1. Are there other settings I need to tweak in the input file to obtain that level of performance? (I saw some bits in the dynamics2d engine help for a "spatial hash" that looked possibly relevant)
2. Would you happen to have the .argos file that you used for the original paper hanging around somewhere that I could take a look at to get an idea of the settings needed?
3. If I wanted to lock the threads used by ARGoS to specific cores, does ARGoS provide a way to do that?

Thanks!
- John

pincy
Site Admin
Posts: 632
Joined: Thu Mar 08, 2012 8:04 pm
Location: Boston, MA
Contact:

Re: How to use multiple physics engines

Postby pincy » Tue Dec 04, 2018 7:25 pm

There are many aspects that go into having real-time performance.

The first and foremost is how you compile your code. Did you use the correct flags? I experimented a little to obtain my results, and settled on -Os and -march=native. Make sure you're using the correct flags.

There are a few settings that can be tweaked to make ARGoS much more performant. These depend on what you're doing - if you share the code with me, I can give you some hints.

Finally, it very much depends on how well your controller code is written and how many sensors/actuators you use.
If I wanted to lock the threads used by ARGoS to specific cores, does ARGoS provide a way to do that?
I'm not sure the scheduler in the kernel allows you to do it. If I'm wrong, it could be an cool feature addition.
I made ARGoS.

jharwell
Posts: 67
Joined: Thu Jan 25, 2018 4:12 am

Re: How to use multiple physics engines

Postby jharwell » Tue Dec 04, 2018 10:53 pm

Here are the compiler flags I use when compiling and linking (for gcc7):

Code: Select all

-O3 -ggdb -Ofast -fno-trapping-math -fno-signed-zeros -march=native -fno-stack-protector -flto
I'm using the footbot proximity, light, motor_ground sensors (though there are others in the input .argos file that I don't use that are used by other controllers--does that matter?).

Here is the ControlStep() of the controller I'm using:

Code: Select all

void crw_controller::ControlStep(void) { saa_subsystem()->actuation()->block_carry_throttle(is_carrying_block()); saa_subsystem()->actuation()->throttling_update( saa_subsystem()->sensing()->tick()); m_fsm->run(); } /* ControlStep() */
The controller basically just runs a state machine (the m_fsm->run() call), which just executes a correlated random walk behavior:

Code: Select all

void random_explore_behavior::execute(void) { rmath::vector2d obs = saa_subsystem()->sensing()->find_closest_obstacle(); saa_subsystem()->steering_force().avoidance(obs); saa_subsystem()->steering_force().wander(); if (saa_subsystem()->sensing()->threatening_obstacle_exists()) { saa_subsystem()->apply_steering_force(std::make_pair(false, false)); saa_subsystem()->actuation()->leds_set_color(utils::color::kRED); } else { saa_subsystem()->actuation()->leds_set_color(utils::color::kMAGENTA); rmath::vector2d force = saa_subsystem()->steering_force().value(); }
The steering force for obstacle avoidance and wandering are calculated as:

Code: Select all

math::vector2d avoidance_force::operator()(const boid &, const math::vector2d &closest) const { if (closest.length() > 0) { math::vector2d avoidance = -closest; return avoidance.normalize() * mc_max; } else { return {0, 0}; /* no threatening obstacles = no avoidance */ } } /* operator()() */

Code: Select all

/* * Only actually apply the wander force at the specified cadence. Otherwise * random perturbations between [-n, n] will sum to 0 (no net wandering) over * time if the perturbations are applied every timestep. */ ++m_count; if (m_count % m_interval != 0) { return {0, 0}; } /* calculate circle center */ math::vector2d velocity; if (entity.linear_velocity().length() <= 0) { velocity = math::vector2d(1, 0); } else { velocity = entity.linear_velocity(); } math::vector2d circle_center = (velocity).normalize().scale(m_circle_distance); /* calculate displacement force (the actual wandering) */ math::vector2d displacement( m_circle_radius * std::cos((m_angle + velocity.angle()).value()), m_circle_radius * std::sin((m_angle + velocity.angle()).value())); double val; if (m_use_normal) { val = m_normal_dist(m_rng); } else { val = -m_max_angle_delta + 2 * m_max_angle_delta * (static_cast<double>(std::rand()) / RAND_MAX); } math::degrees perturbation(std::fmod(math::to_degrees(m_angle).value() + val, m_max_angle_delta)); m_angle = math::to_radians(perturbation); /* * Wandering is a combination of the current velocity, projected outward a * certain distance, and a displacement calculated at that point. * * There can be discontinuous jumps from a small positive angle for the wander * force to a large negative angle between timesteps and vice versa which play * havoc with robots' ability to compensate. * * So, compute the angle indirectly using sine and cosine in order to account * for sign differences and ensure continuity. */ double angle = std::atan2(displacement.y() - circle_center.y(), displacement.x() - circle_center.x()); double angle_diff = angle - circle_center.angle().value(); angle_diff = std::atan2(std::sin(angle_diff), std::cos(angle_diff)); if (std::fabs(angle_diff - m_last_angle) > M_PI) { angle_diff -= std::copysign(2 * M_PI, angle_diff); } m_last_angle = angle_diff; math::vector2d wander((circle_center + displacement).length(), math::radians(angle_diff)); return wander.normalize() * m_max;

Lastly, here is the input file I'm using (project-specific parameters omitted for clarity):

Code: Select all

<argos-configuration> <framework> <system threads="16" /> <experiment length="5000" random_seed="33" ticks_per_second="5" /> </framework> <controllers> <crw_controller id="ffc" library="libfordyca"> <actuators> <differential_steering implementation="default" /> <leds implementation="default" medium="leds" /> <range_and_bearing implementation="default" /> </actuators> <sensors> <footbot_proximity implementation="default" show_rays="true" /> <footbot_light implementation="rot_z_only" show_rays="false" /> <footbot_motor_ground implementation="rot_z_only" /> <range_and_bearing implementation="medium" medium="rab" /> <battery implementation="default" /> </sensors> <params> </params> </crw_controller> </controllers> <loop_functions label="depth0_loop_functions" library="libfordyca"> </loop_functions> <arena center="16.0, 8.0, 1" size="32, 16, 2"> <floor id="floor" pixels_per_meter="50" source="loop_functions" /> <box id="wall_north" movable="false" size="32, 0.1, 0.5"> <body orientation="0,0,0" position="16.0, 16, 0" /> </box> <box id="wall_south" movable="false" size="32, 0.1, 0.5"> <body orientation="0,0,0" position="16.0, 0, 0 " /> </box> <box id="wall_east" movable="false" size="0.1, 16, 0.5"> <body orientation="0,0,0" position="32, 8.0, 0" /> </box> <box id="wall_west" movable="false" size="0.1, 16, 0.5"> <body orientation="0,0,0" position="0, 8.0, 0" /> </box> <distribute> <position max="30, 15, 0" method="uniform" min="1,1,0" /> <orientation max="360,0,0" method="uniform" min="0,0,0" /> <entity max_trials="1000" quantity="1024"> <foot-bot id="fb"> <controller config="ffc" /> </foot-bot> </entity> </distribute> </arena> <media> <range_and_bearing id="rab" /> <led id="leds" /> </media> <physics_engines> <dynamics2d id="dyn2d0"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="0.0, 0.0" /> <vertex point="8.0, 0.0" /> <vertex point="8.0, 4.0" /> <vertex point="0.0, 4.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d1"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="8.0, 0.0" /> <vertex point="16.0, 0.0" /> <vertex point="16.0, 4.0" /> <vertex point="8.0, 4.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d2"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="16.0, 0.0" /> <vertex point="24.0, 0.0" /> <vertex point="24.0, 4.0" /> <vertex point="16.0, 4.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d3"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="24.0, 0.0" /> <vertex point="32.0, 0.0" /> <vertex point="32.0, 4.0" /> <vertex point="24.0, 4.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d4"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="24.0, 4.0" /> <vertex point="32.0, 4.0" /> <vertex point="32.0, 8.0" /> <vertex point="24.0, 8.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d5"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="16.0, 4.0" /> <vertex point="24.0, 4.0" /> <vertex point="24.0, 8.0" /> <vertex point="16.0, 8.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d6"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="8.0, 4.0" /> <vertex point="16.0, 4.0" /> <vertex point="16.0, 8.0" /> <vertex point="8.0, 8.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d7"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="0.0, 4.0" /> <vertex point="8.0, 4.0" /> <vertex point="8.0, 8.0" /> <vertex point="0.0, 8.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d8"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="0.0, 8.0" /> <vertex point="8.0, 8.0" /> <vertex point="8.0, 12.0" /> <vertex point="0.0, 12.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d9"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="8.0, 8.0" /> <vertex point="16.0, 8.0" /> <vertex point="16.0, 12.0" /> <vertex point="8.0, 12.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d10"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="16.0, 8.0" /> <vertex point="24.0, 8.0" /> <vertex point="24.0, 12.0" /> <vertex point="16.0, 12.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d11"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="24.0, 8.0" /> <vertex point="32.0, 8.0" /> <vertex point="32.0, 12.0" /> <vertex point="24.0, 12.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d12"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="24.0, 12.0" /> <vertex point="32.0, 12.0" /> <vertex point="32.0, 16.0" /> <vertex point="24.0, 16.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d13"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="16.0, 12.0" /> <vertex point="24.0, 12.0" /> <vertex point="24.0, 16.0" /> <vertex point="16.0, 16.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d14"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="8.0, 12.0" /> <vertex point="16.0, 12.0" /> <vertex point="16.0, 16.0" /> <vertex point="8.0, 16.0" /> </sides> </boundaries> </dynamics2d> <dynamics2d id="dyn2d15"> <boundaries> <top height="1" /> <bottom height="0" /> <sides> <vertex point="0.0, 12.0" /> <vertex point="8.0, 12.0" /> <vertex point="8.0, 16.0" /> <vertex point="0.0, 16.0" /> </sides> </boundaries> </dynamics2d> </physics_engines> </argos-configuration>
As far as locking threads to cores, there is not a POSIX/standard way to do it, but on linux you can use pthread_setaffinity_np() to set which CPUs a thread is eligible to run on, and sched_affinitity() to set which cores a process is eligible to run on (there are different calls to do it on OSX). Providing options for one or both of these in the .argos input file, or exposing the std::thread handle (at least I assumed that's what ARGoS uses to create/manage threads) so that users can set them programmatically would definitely be a great feature to add. On shared systems where ARGoS may have to contend with other high performance software as it runs this would help to make it as efficient as possible by minimizing overhead due to thread context switching/cache invalidation when threads switch cores.

I think this is something relatively simple that could be added, and I would like contribute to ARGoS and add it. How can I get started on doing so?

- John

jharwell
Posts: 67
Joined: Thu Jan 25, 2018 4:12 am

Re: How to use multiple physics engines

Postby jharwell » Wed Dec 12, 2018 9:47 pm

Also, I've been playing with disabling the FSM the robots are using altogether (i.e. the ControlStep() now just returns without doing anything), as well as removing all the bits in the loop functions that I added, and I have been able to get close to real-time performance with a swarm of 1024 robots (within about 20%), but have not been able to get closer than that.

What are some of the ARGoS settings I should be looking at tweaking to get that extra boost?

- John

pincy
Site Admin
Posts: 632
Joined: Thu Mar 08, 2012 8:04 pm
Location: Boston, MA
Contact:

Re: How to use multiple physics engines

Postby pincy » Thu Dec 13, 2018 4:24 am

Partly is the compilation flags - I found that using -Os is better than -O3 because the smaller binary size works well with large simulations.

A few tweaks that improve performance:

1. If you're using the range-and-bearing sensor with a range R in an arena of width W and depth D, you could optimize the size of the internal grid of the RAB medium. Calling Cx the number of cells along the x axis and Cy the number of cells along y, then
  • Cx = ceil(W / (2*R))
  • Cy = ceil(D / (2*R))

Code: Select all

<media> <rab_medium id="rab" grid_size="Cx, Cy, 1" /> </media>
2. In the same vein, you could optimize the internal storage of the LEDs. With the foot-bot, for instance, the LEDs are distributed along the external circumference of the robot. The radius of the foot-bot is R = 8.6 cm [1], so using the same equations as above you can set

Code: Select all

<media> <led_medium id="led" grid_size="Cx, Cy, 1" /> </media>
3. Another tweak you can try concerns thread scheduling. There are currently two available thread scheduling approaches: "balance quantity" and "balance length". The first assigns the same number of tasks to every thread; it works best when the tasks are similar in required computational cost. The second assigns tasks according to which thread has worked less, so while a long task is performed by a thread, the other threads can go through shorter tasks in parallel.

4. Above everything else, your code must also be very performant, because that's where most of the computation tends to go. The math in the code you linked above can be significantly optimized for speed.

[1] https://github.com/ilpincy/argos3/blob/ ... ty.cpp#L31
I made ARGoS.


Return to “How to... ?”