IHMC- Quarterly Report Jan - March 2006 1) Progress Against Planned Objectives Our planned objectives from the previous reporting period and their status is as follows: a) Finish the Version1 algorithm and test it on various Terrains. Done. Version1 algorithm was finished and tested on various terrains with holes, steps, and rolling terrain. b) Start the Version2 algorithm and incorporate some of our development goals. Version2 has been started and some of the goals accomplished. We are still finishing the development of Version2. c) Develop a hierarchical learning algorithm for a Footstep Generator over two-dimensional Terrain in which a “swath” is first determined using a low-resolution map of the Terrain and then the Footstep sequence is determined inside that swath using a high-resolution map of the Terrain. Still under development. d) Use Reinforcement Learning techniques in the Stance Generator. Done. We implemented a Dynamic Programming Stance Generator that seems to work quite well. e) Develop a control algorithm that does not use traditional stiff trajectory tracking, but instead uses more compliant force-control methods. Think about the feasibility of using such methods on the LittleDog robot, which does not have force-controllable actuators. We have developed the mathematics for such a technique and tested it in simulation and on the real robot. In simulation it works very well. On the real robot, as expected, it doesn't work so well due to joint friction. However, in some cases it seems to work ok and we are now investigating the potential of using a combination of force control and position control in further algorithms. f) Develop a Trot-Walking algorithm for fast walking on smooth Terrain. Still under development. We have significantly sped up our crawl gait walk and will continue working on a trot gait to increase the speed. g) Develop user input methods for selecting Footsteps by clicking on them in the Simulation GUI. Still under development. We have implemented some other GUI functionality, such as viewing the kinematically reachable area of the center of mass, given the footstep locations and the use of a MIDI slider panel for inputting user input to the algorithm. h) Continue the development of our Reinforcement Learning Toolbox. The Reinforcement Learning toolbox, particularly the Dynamic Programming tools are now quite stable and work quite well. Since DP has been working so well for us, we have not required the use of the other methods too much so far. i) Get the Simulated LittleDog to walk over fairly rough two-dimensional Terrain. Under development. j) Add noise to the simulations. We have not done this yet, since the ground springiness adds a sort of noise and we are just starting to look into what we can expect from the LittleDog and Motion Capture sensors. k) Develop a methodology for measuring the performance of a given algorithm. Right now we are using the government testing formula for measuring performance, but may revisit this later. l) Acquire our LittleDogRobot and get it working. Done. ----------------------------------------------------------------------------- 2) Technical Accomplishments A) Summary: During this period, we developed a total Center of Mass calculator for the LittleDog; developed an iteratve Inverse Kinematic method for positioning the Center of Mass (as opposed to the Center of Body); developed a kinematic reachability and position plotter; finished our Version1 simulation algorithm and got the simulated LittleDog to walk over rolling terrain, steps, and gaps; acquired our LittleDog robot and Motion Capture system; set up the truss system and cameras for the Motion Capture system; reworked the forward and inverse kinematics for the new LittleDog configuration; developed a fast reactive crawl gait walking algorithm; and implemented the low-level software for using our development environment, software tools, and GUIs with the real robot. B) Center of Mass calculator and iterative Inverse Kinematics Algorithm The center of mass of the LittleDog robot is a function of the mass of each link and the joint angles of the joints. We derived, implemented, and tested the math for calculating the COM relative to the center of the body. For stability of the robot, it is important to be able to position the COM relative to the feet. However, the joint angles are needed to determine the location of the COM and the joint angles are unknown. Therefore, we developed an iterative inverse kinematics procedure: 1. Guess a location of the body center based on the desired COM location. 2. Calculate the resulting joint angles. 3. Calculate the resulting COM. 4. Compare the result of step 4 with the desired COM. If the error is less than a threshold, stop. If not, refine the guess location of the body center and go back to step 2. Using this algorithm, we can position the COM within 0.0005 m of the desired position in typically two iterations. C) Kinematic Reachability and Position Plotter. The positions that the center of the body can be moved to are a function of the foot locations. Given a set of foot locations, the center of the body must lie in an closed volume of space. To help visualize this volume we developed a Kinematic Reachability plotter that determines and graphs this volume given a set of footsteps. For a footstep sequence, these volumes must overlap so that the robot can kinematically reach each set of footsteps. Originally, we computed this volume by computing the Inverse Kinematics for a grid of points in the volume and checked whether an inverse kinematic solution existed. However, this check is very computationally expensive so we have recently been determining much quicker checks for computing which positions are reachable. D) Version1 Simulation Algorithm The main addition to the Version1 simulation algorithm from the previous reporting period is the inclusion of a Dynamic Programming Stance Generator. Given a sequence of Footsteps, the Stance Generator determines a sequence of Stances (body x, y, z, roll, pitch, and yaw). For this algorithm, we confined the roll and yaw to be zero and the pitch to match the angle from the front feet to the hind feet. The height above the ground was set to be a constant height above the average of the feet locations. x and y were then the state variables to be optimized. The action vector was the distance to move in x and in y. To increase efficiency, we precomputed all of the state-action-reward transition pairs and stored them in a table. States were pruned that were not kinematically reachable. Then Dynamic Programming was used on this table to find the optimal Stances. For a typical terrain, 3 minutes of computation time was required to find a sequence of Stances over the terrain. The majority of the computation time was required for determining if a state was kinematically reachable. Once the table was precomputed, only 3 seconds or so were required to solve for the optimal. We are currently working on speeding up the test of whether a Stance is kinematically reachable, and believe we can speed the Dynamic Programming Stance Generator up by a factor of 10 or so. We tested the Version1 algorithm on a number of simulated terrains that included rolling terrain, holes and gaps, and steps. Gap sizes were on the order of 5cm and steps were on the order of 4cm. The walking was extremely slow as we were concentrating on static walking and accurate foot placement. The Dynamic Programming planners were very good at finding a sequence of steps across the terrain, often requiring the body to move backward a little in order to allow a foot to clear a gap. We are very pleased with this Dynamic Programming algorithm and will continue to push its development on 2D terrain. E) Fast Reactive Crawl Gait Algorithm For moderate terrain, we developed a purely reactive static crawl gait algorithm. This algorithm does not use information about the terrain to walk. It only uses the internal sensors and foot sensors to walk over the terrain. The joints of each leg are controlled by a Cartesian velocity controller. For each leg, a desired velocity of the foot is determined. from the desired velocity, desired position, and time step, the next desired foot position is determined: x_des(k+1) = x_d(k) + v_des(k) * delta_t Where x_des(i) is the desired hip to foot position vector at the ith time step, v_des is the desired velocity vector of the foot, and delta_t is the control loop time step. Using inverse kinematics, the joint angles for the leg are determined from x_des. The desired velocity for the leg depends if it is a stance leg or a swing leg. For the stance leg, the desired velocity comes from a series of controllers. The output of each controller is leg velocities, which are then added together. a) The stance height controller tries to have the z component of the hip to foot vector (in body coordinates) equal to a desired value. If it is not, then it moves the foot in the desired direction. The velocity that this controller outputs is low, and thus the leg will drift to the desired position over the course of a second or so; b)The center of mass (COM) position controller moves the COM to a position relative to the feet. It does this be determining how the COM should move, and then moves all of the feet in the opposite direction. The desired COM position is determined by the triangle formed by the three feet, with the to be swing leg not included. The target COM position is adjusted slightly forward or backward depending on the pitch of the body. Once the COM is in position, the swing leg starts swinging. When a leg is swinging, the position of the swing leg is determined to regulate the stance length and width and the stepping x and y distances. The X step length and the Y step length are user input parameters. The X value causes it to move forward and back, and the Y value causes it to move left and right. The Stance length controller and the Stance width controller control the spacing of the feet relative to each other. This makes sure that the feet are spread out according to user set desired values. When swinging, the leg moves up by a set amount (step height), then it moves a displacement from its starting point in the X and Y directions by Xdes and Ydes. When it has moved this amount, it moves down until the foot switch registers a contact. The reactive walker has a fixed gait sequence of: hind right, front right, hind left, front left, and then back to hind right again. When the swing leg touches down, the next swing leg is picked from the list, and a new COM target position is determined. The COM moves to position, the leg swings, and the process repeats. Using this controller, we have achieved walking on flat ground and over stairs with the simulated LittleDog. F) Low level software for LittleDog robot integration. To stay consistent with our simulation work and to provide, in our view, the best development environment, we have developed low level software for using the same development tools on the LittleDog robot as we have been for simulation. We have figured out how to develope code for the robot in both C++ and in Java. We have created base classes so that the real robot and the simulated robot are fairly seemlessly integrated. In fact, we now use the exact same control file on the real robot as on the simulated robot. We are now expanding these classes to include information from the IMUs and the motion capture system. We also use the simulation GUI as a GUI for the real robot. In that way we can see real time plots of data of any variable while the robot is running, rather than wait for the end of a run to visualize data. We have also gotten a MIDI slider board to work on the LittleDog host computer so that we can get user input while the robot is walking. This is useful for things like tuning parameters on the fly. ----------------------------------------------------------------------------- 3) Improvements to Prototypes None. ----------------------------------------------------------------------------- 4) Significant Changes to Technical Approach None. ----------------------------------------------------------------------------- 5) Planned Activities a) Finish incorporating the Motion Capture and IMU signals for use in our control algorithms. Gain familiarity with both systems. b) Continue investigating compliant force control techniques and how they can be used on the LittleDog, perhaps in combination with position control techniques. c) Develop a Trot-Walking algorithm for fast walking on smooth Terrain. d) Get the Simulated LittleDog to walk over fairly rough two-dimensional Terrain. e) Get the real LittleDog to walk over increasingly rough terrain. f) Develop a hierarchical learning algorithm for a Footstep Generator over two-dimensional Terrain in which a swath is first determined using a low-resolution map of the Terrain and then the Footstep sequence is determined inside that swath using a high-resolution map of the Terrain. g) Develop user input methods for selecting Footsteps by clicking on them in the Simulation GUI. ----------------------------------------------------------------------------- 6) Issues or Concerns None.