Home / GPS&GIS / Using a vision sensor system for auto-guidance testing

Using a vision sensor system for auto-guidance testing

Auto-guidance (also called auto-steering) systems represent  a rapidly expanding technology in precision agriculture that is  based on the use of global navigation satellite system (GNSS) receivers to perform field operations in a strict geometrical relationship with a previous travel path or other predefined geographical coordinates, without direct inputs from an operator.

Although auto-guidance systems available to producers have  different levels of operation accuracy as well as sensor configurations and interfaces, their performance is frequently associated with an anticipated level of auto-guidance error, usually  referred to as cross-track error (XTE). This error can be attributed to numerous uncertainties, including: (1) geographic positioning errors; (2) vehicle dynamics; (3) the implement tracking  behind the vehicle; (4) the field environment (slopes, soil condition, etc.). Manufacturers of auto-guidance systems publish  claims that rely on a variety of different test procedures, and as  a result, consumers cannot use marketing information to compare the performance of different products. Therefore, there is a  need to develop a standardized procedure to test and report the performance of GNSS-based auto-guidance systems.

The first step in testing GNSS-based equipment involves evaluation of the static performance of GNSS receivers by placing the antenna in a fixed georeferenced location (ION, 1997)  and logging measurements made by the receiver. Agricultural  operations are dynamic in nature; therefore, tests of GNSS receivers used in agriculture should be performed while in motion. Stombaugh et al. (2002) and later Stombaugh et al. (2008)  provide general guidelines for a dynamic test. Two main dynamic GNSS receiver testing methods were defined: (1) fixturebased testing, which involves mounting the GNSS receiver on a platform that is operated along a fixed path with known geographic coordinates and (2) vehicle-based testing, in which the tested set of receivers is placed on the top of a vehicle along with

a superior performance measurement system, possibly a realtime kinematic (RTK) GNSS receiver. Advantages of fixturebased testing include the ability to calculate errors with respect  to the actual (surveyed) geographic coordinates and the repeatability of the testing procedure. The advantage of vehicle-based testing is that it can represent actual field operations.

Though fixture-based testing cannot be used to evaluate the actual performance of a vehicle operated using a GNSS receiver, Han et al. (2004) implemented a vehicle-based approach to test eight commercially available DGPS receivers from four different manufacturers with five alternative differential correction services. All eight tested GPS receivers were mounted simultaneously on a test vehicle at least 1 m apart from each other to reduce possible signal interference. An RTK-level GPS receiver was mounted in the center of the test platform to provide the vehicle reference positions. The vehicle was manually driven on a travel path as straight as possible in the north-south direction, with each test consisting of six parallel passes approximately 305 m (1000 ft) long. The desired pass-to-pass spacing was 6.10 m (20 ft). Offtrack errors were determined as the root mean squared difference between the horizontal position determined by the reference receiver (with the appropriate offset compensations) and the tested receivers. Pass-to-pass error was defined as the difference between corresponding off-track errors. It was noted that travel speed might play an important role in quantifying receiver accuracy since at lower speeds, the pass-to-pass average errors tended to be larger. Han et al. (2004) associated the increase in pass-topass error with the longer time needed to complete the test course  when moving slower. However, due to data limitations, complete analysis of the effect of travel speed was not conducted.

A similar approach can be used to test the performance of navigation aids, known as light bar systems. Light bar systems assist the operator of an agricultural vehicle in steering it according to GNSS position estimates. Buick and Lange (1998) and later Buick and White (1999) compared the efficiencies of foam

marker and GPS-based light bar guidance systems. Field efficiencies were determined by measuring the actual areas of skips and overlaps for different ground speeds and offline distances  (based on vehicle track records). In another study, Ehsani et al.  (2002) tested different GPS-based light bar systems by mounting them on the roof of a tractor and driving nine swaths parallel to a pre-set A–B line. In both cases, an RTK receiver was used to determine the actual travel path.

The testing of auto-guidance systems has become the latest  challenge when it comes to the GNSS-based operation of agricultural vehicles. The measurement system for test instrumentation  must have at least ten times greater accuracy than the system being tested (ION, 1997). This means that for auto-guidance systems equipped with meter and decimeter-level GNSS receivers, a centimeter-level sensor, such as an RTK-level GNSS receiver, can be used. However, since many advanced auto-guidance options employ centimeter-level GNSS receivers, an appropriate test system  should be capable of making millimeter-level measurements.

Harbuck et al. (2006) employed optical surveying equipment  to track vehicle motion without the involvement of GNSS-based  equipment. A rugged 360-degree tracking prism was mounted  to the towing hitch on the rear of the tractor. Position data was  recorded using a total station equipped with a special function  that made it possible to follow the moving prism by the use of  servo motors in the total station base. During each test, the tractor was operated through a straight pass using the auto-guidance system, and the relative position of the tractor hitch was  continuously recorded. The claimed 5-mm measurement error  of the total station was applicable under ideal conditions, but  this error increased to 20 m during the test. Consequently, the  order of magnitude required for greater accuracy by the measurement system was no longer valid.

Adamchuk et al. (2007) developed a linear potentiometer array that measured the horizontal position of a reference cart  perpendicular to the direction of travel as it repeatedly passed over a series of stationary metal triggers installed on the surface of the pavement used for testing. The system had an approximate resolution of 20 mm and did not rely on a GNSS signal. Although both methods are suitable for many non-RTK-based options, testing auto-guidance systems with a claimed accuracy of around 20 mm would require a more precise solution.

The objectives of this research were (1) to develop instrumentation and test methodology for measuring relative XTE with millimeter-level accuracy; (2) to evaluate the method developed by comparing the performance of tractors with auto-guidance systems operated at various travel speeds; (3) to recommend a test procedure for measuring pass-to-pass and long-term relative XTE in a repeatable manner.

2. Materials and methods

2.1. Instrumentation development

Testing auto-guidance systems requires a method of measurement that is accurate enough, yet easy to use and adaptable to multiple situations. After a number of options involving different optical referencing techniques were considered, the final choice was the machine vision approach. Various machine vision sensors are used extensively in industry for real-time monitoring of product dimensions and quality control. Following a test concept pursued by Adamchuk et al. (2007), the vision sensor was mounted on the tested vehicle to monitor a permanent reference line on the pavement below. As the vehicle moved along the test track, it was possible to measure the relative position of the tested vehicle with respect to the permanent reference line in every location along the test track.

Since the test was focused on auto-guidance systems with RTK-level GNSS receivers, a 1.2-m field of view was assumed appropriate so that the line remained visible to the sensor during the entire test. To achieve the 2-mm sensor resolution required by the 20-mm claimed accuracy would involve a 600-pixel array (1200 mm/2 mm) in the horizontal direction (perpendicular to the direction of travel). A Cognex In-Sight DVT 545 high-speed vision sensor with an internal processor  (Cognex Corporation, Natick, MA)* and a NAV LFC-9F1B 9-mm  lens was considered sufficient. The sensor had a 640 × 1048 pixel array with a 26° field of view. This provided approximately 1.2-mm resolution at the testing surface when mounted 1.5 m above the ground and pointed downward. The sensor was capable of automatically adjusting exposure and aperture settings for varying lighting conditions and could process images at the rate of

approximately 30 frames/s. The vision sensor calibration, crosstrack position measurements, and other adjustments were made using Intellect™ (Cognex Corp., Natick, MA) software.

Relative position measurements performed with the vision sensor were synchronized with geographic locations to allow the matching of measurements obtained during different passes.

An additional GNSS receiver was used to obtain geographic longitude and latitude, time, and GNSS signal quality for further data processing. Data acquisition was accomplished using  a specially developed LabVIEW® (National Instruments, Inc., Austin, TX) interface.

2.2. Test procedure development

The test procedure developed was based on a typical field  operation in which a series of back and forth parallel passes across a certain distance are performed. At the end of each pass, the vehicle is turned around and returns on a path adjacent to the previous pass with an offset equal to the fixed width of the

implement (swath width). In this case, relative XTE can be defined as the difference between the desired and actual swath widths. If the distance between two passes is less than the swath width, an overlap will occur; a distance greater than the swath width produces a skip. Pass-to-pass error of auto-guidance is

defined as the relative XTE between two consecutive passes that occur within a 15-min time interval. Long-term auto-guidance error is defined as the relative XTE between two consecutive passes that occur more than 1 h apart with dissimilar GNSS satellite configurations in the sky.

To accommodate these definitions, each test consisted of three test runs with three passes about 7.5 min long made in alternating directions. An appropriate test location should have a pavement surface that would not change over time and could be replicated in various geographic areas. Since tractor performance testing is typically done on concrete pavement, the same approach was used in developing an auto-guidance system test procedure. The concrete tractor test track of the Nebraska Tractor Test Laboratory (NTTL, Lincoln, NE) was selected to also serve as the test track for the auto-guidance systems testing. The track consists of two east-west oriented straight passes separated by 39.9 m (131 ft). Both passes are relatively level, with the total length of the central line around the track being approximately 615 m (2018 ft). Each straight pass of the track was 6.7 m (22 ft) wide with an expansion seam in the middle, which was designated as the permanent reference line. Theoretically, the east–west direction of the track presented the most challenging autoguidance operation environment from the viewpoint of GNSS performance, as the positioning error associated with latitude is typically greater than the longitudinal error for this location.

To adapt the ideal (back and forth) field operation pattern to the geometry of the test track, the test trial consisted of a sequence of counterclockwise, clockwise and counterclockwise laps around the track.

The initial A–B line was established along the northern pass and the auto-guidance equipment was set with a 39.9-m swath width. The test tractor was operated in auto-guidance mode along each of the two passes. During each pass, the relative location of the tractor’s representative vehicle point (RVP) with respect to the reference line was measured using the visual sensor. For each location around the track, the difference between these relative position measurements (adjusted for the direction of travel) was used to define relative XTE.

For the test vehicle, the decision was made to use the most common platform on which auto-guidance systems are installed. Mechanical front wheel assist tractors with dual reartires in the range of PTO power from 110 to 220 kW (150 to 300 hp) were selected. The drawbar hitch pin hole was designated

as the RVP for these vehicles. The vision sensor was rigidly mounted to the chassis of the tractor with the lens pointed downward so that the field of view was centered on the drawbar hitch pivoting location (pin hole). To calibrate the vision sensor, a Cognex® 100-mm calibration grid was centered under the hitch pin hole with the horizontal axis parallel to the rear axle of the tractor.

To set an A–B line on the northern side of the track, the tractor was manually operated close to the expansion seam line at least an hour before the test. At the beginning of each test run, the tractor was located at the northeast corner of the track facing west (ready to travel in a counterclockwise direction around the track). The data acquisition system was started, the tractor moved forward, and the auto-guidance system was engaged. The tractor traveled along the north side of the track with a swath width of 0 m with respect to the original A–B line until it reached the end of the northern pass. At the curve, the operator manually steered the tractor counterclockwise around the western curve of the track and lined it up with the expansion seam of the southern portion of the track. While entering the southern pass, the auto-guidance system was re-engaged with a swath width of 39.9 m from the original A–B line. The autoguidance mode remained engaged along the southern pass of the track until the tractor reached the eastern end. The operator again took control and manually steered the tractor around the curve. At this point, depending on the lap number and the number of laps required by the test speed,

the tractor either turned around to travel in a clockwise (CW) direction or continued travel to complete the number of counterclockwise laps required for that travel speed. The tractor traveled at speeds ranging from 0.5 to 5.0 m/s. To account for increased travel distances due to higher speed, specific numbers of laps in the counterclockwise, clockwise, and counterclockwise direction were performed so the travel time in one direction was just greater than 7.5 min. Table 1 shows the number of laps in each direction required for each tested travel speed.As the tractor entered into a new straight pass, the autoguidance mode was engaged before the end of the curve unless a change of travel direction from clockwise to counterclockwise was made. In this scenario, the auto-guidance system was engaged before a set point on the northern side of the track. Data taken before this point were excluded from the analysis.

To process the data, average valid relative position and time measurements were obtained for each 1-m segment of the track. The local projection of geographic position measurements obtained using an additional GNSS receiver located above the visual sensor was based on the WGS-84 ellipsoid and used to assign each measurement to a corresponding test track segment. Relative pass-to-pass XTE terms were found by comparing relative position measurements (visual sensor output) between two collocated points from passes in a single test run that were obtained during travel in opposite directions with revisit time under 15 min. Long-term error was also detected by comparing cross-track position measurements between passes obtained during travel in opposite directions but from different test runs.

From unsigned values of relative XTE, cumulative distributions were constructed with 95% errors identified, as a 95% absolute error is the most frequently used value referred to in promotional literature. Mean values of signed XTE were calculated  to determine bias in the auto-guidance system. Unfortunately, because of the potential to obtain non-zero mean of a signed error distribution caused by either temporal positioning offset or non-symmetric response of steering controller, it is not appropriate to conduct conventional variance analysis to compare results from different tests. Therefore, the mean and the standard deviation of the normally distributed signed errors were used to numerically (5,000 random points) assess the probability of absolute errors being greater for one test with respect to another test.

For illustration, every reported test result has been compared to a frequently cited by equipment manufacturers claim of 95% error within 25.4 mm (1 in.). Generally, such a claim should mean a normal error distribution with zero mean and 12.7 mm standard deviation, which has been denoted as a benchmark claim.

Variations in pass entrance errors caused differences in the lengths of transition to steady-state operation, so two different sets of data with transition regions of different lengths were analyzed. These were labeled as inner and outer marks. The inner marks were located 30.5 m (100 ft) inside the outer marks to allow extra distance for the auto-guidance system to reach equilibrium before entering the region where the data were considered valid for the purpose of testing. While the inner marks allowed a longer distance for the auto-guidance steering control to reach steady-state operation, the outer marks

provided amore complete revisit time distribution as more data points were considered part of the valid dataset.

2.3. Test system evaluation (Test A)

For Test A, the three selected travel speeds were: 1.0, 2.5, and 5.0 m/s. A John Deere 8520 tractor with a Trimble AgGPS RTK Autopilot™ system (Nav 2 controller) was provided by Trimble Navigation Limited (Sunnyvale, CA). The tractor was outfitted with the vision sensor above the RVP, as well as with an additional georeferencing receiver (OutbackS, Outback Guidance, Hiawatha, KS). The test served as the initial trial of what could become a standardized test protocol. After data processing, a follow-up test (Test B) to investigate the travel speed effect was conducted.

2.4. Travel speed testing (Test B)

Many field operations are done with travel at the speed of approximately 2.5 m/s (5.6 mile/h), but some operations (e.g., spraying) may require travel speeds of approximately 5.0 m/s (11.2 mile/h) or higher. Specialty crops may require travel speeds as low as 0.1 m/s (0.22 mile/h). Unfortunately, the tractor available for this second test was not capable of operating in auto-guidance mode at 0.1 m/s. Therefore, to observe the effect of travel speed on auto-guidance error, eight speed settings were selected for Test B: 0.5, 1.0, 2.5, 3.0, 3.5, 4.0, 4.5, and 5.0 m/s. For this testing, a UNL-owned John Deere 7820 tractor equipped with a Trimble AgGPS RTK Autopilot™ system (Nav 1 controller) and an AgLeader Insight (AgLeader Technology, Inc., Ames, IA) terminal were used. The same test scheme was used, with the tractor traveling a fixed number of laps around the track at a fixed travel speed. However, the 0.5 m/s travel speed did not allow completion of a full lap around the track, so only the northern side of the track was used instead. For Test B, the georeferencing receiver was an RTKlevel Trimble AgGPS 442 (GPS/GLONASS).

3. Results and discussion

3.1. Test system evaluation (Test A)

Variations in the number of occurrences of revisit time intervals were due to the effect of distance-based segmentation of the test track. Higher travel speeds had more revisit occurrences as compared to slower travel speeds. Although segmentation of the data on a time basis might be a more appropriate approach in terms of revisit time distributions, a distance-based segmentation approach had greater practical value since it involves the potential for skips and overlaps that can occur when operating under field conditions.

During the analysis process, as noted above, greater errors were observed at the beginning of each northern or southern pass, caused by the variation in manual steering to align the tractor with the track line before engaging the auto-guidance mode of operation. Therefore, the inner and outer marks were used to run separate analyses. Although the system was engaged in auto-guidance mode before crossing the outer marks, using the inner marks to designate valid data collection areas allowed for a longer steering stabilization distance. For example, significantly higher differences in relative tractor position between passes in the opposite direction occurred at the entrances to the northern and southern portions of the track. At slower travel speeds (1.0 and 2.5 m/s) these differences were negligible.

From the summary, it appears the average signed errors were only a few millimeters, except for the long-term error when operating at 5.0 m/s and using the outer marks to process the data (26 mm). This provides a foundation for assuming that the auto-guidance system did not have a steering bias and the error followed a normal distribution centered near  zero. The exception for the high speed was caused by the difference in the manual entrance methods for the northern or southern test pass during different test runs. It appears the auto-guidance controller converged more slowly to a straight line at the beginning of the test, which signifies some differences in terms of the timing of system engagement and angle of entering the straight passes. On the other hand, this bias was not found for any consecutive test run (pass-to-pass error) or when using the inner marks Based on Test A results, the cumulative 95% auto-guidance error values for the lower travel speeds of 1.0 and 2.5 m/s confirmed the 20-mm accuracy claimed by the manufacturer. The results also indicated that travel speed caused auto-guidance error to increase when operating at 5.0 m/s, which suggested the need for follow-up testing. These increases resulted in 60–80% probability of the errors being higher than the established benchmark, whereas 50% probability means no difference at all.

Also, it was noted that the probability was less than 50% for 1 and 2.5 m/s travel speed, which means that the guidance error was lower than 25.4 mm (1 in.) 95% of time. Yet, none of these differences were found significant at  = 0.1 (less than 10 or more than 90% probability).

3.2. Travel speed testing (Test B)

Although every effort was made during this test to assure the same turning path was followed, the effect of significant positioning error in the transition from manual to steady-state autoguidance steering operation at the 5.0 m/s travel speed was observed (as during Test A). This phenomenon was once again not significant for the slow and medium travel speeds.

In this case, the mean signed errors were less than their standard deviations for all travel speeds (maximum at 6.9 mm with 29.4 mm standard deviation for long-term error estimated using outer marks when operating at 3 m/s). This confirms the assumption there was no significant steering bias in the system. However, the cumulative distributions of the unsigned XTE terms indicate an increase of auto-guidance error values for Test B in comparison to Test A.

At 1.0 and 2.5 m/s travel speeds, the 95% auto-guidance error estimates increased from approximately 20 mm to approximately 50 mm. For the 5.0 m/s travel speed, this increase was from about 40 mm to as much as 90 mm (125%). The probability that the guidance error is greater than the benchmark claim rose to 62–83%. This error increase over Test A could be due to multiple factors: (1) differences in the auto-guidance systems; (2) the quality of the positioning signal; (3) the system set-up and vehicle dynamics. Test A involved a larger tractor with the most upto-date control hardware and software, while Test B was performed with an older version of the control hardware. The Test A system was operated and fine-tuned by company representatives according to the procedures available to the consumer but usually set once and not readjusted thereafter. The Test B tractor was obtained from an actual production farm, fitted with the measurement system and tested without the recommended calibration procedure. Despite these differences, for the medium travel speed tests processed using inner marks, the estimated 95% error measurements were still below 50 mm.

Again, it was noted that the inner marks provided lower 95% error estimates compared to the outer marks. The difference between pass-to-pass and long-term auto-guidance error  terms was below 10 mm for most travel speeds, which confirms the long-term repeatability of the RTK-level GNSS.

Levels of probability that an individual error measurement for each combination of auto-guidance systems and travel speeds is greater than an individual measurement for any other such combination (pass-to-pass errors and inner marks only). It appeared the greatest increase occurred when switching from the travel speed of 4.5 to 5.0 m/s. This increase could be due to vehicle dynamics and the need for a longer distance to enter the steady-state operation mode. At slow travel speeds (0.5 and 1.0 m/s) the performance of the autoguidance system did not appear to be different from performance with a travel speed of 2.5 m/s. Therefore, it seems reasonable to focus on two or three travel speeds when developing the standardized test procedure. For instance, the standardized test could include a medium (2.5 m/s) and a fast (5.0 m/s) travel speed, for which autoguidance error was found to increase 50–100%. Lower than 1.0 m/s speeds could be considered in special cases when very slow field operations are anticipated.

4. Conclusions

In this study, a high-speed vision sensor-based system was used to quantify the accuracy of GNSS-based auto-guidance systems. Pass-to-pass and long-term cross-track errors were defined as the ability of the system to repeat the same pass in an opposite direction within short (15 min) and long (several hours) time periods. A comparison of three different travel speeds (Test A) and eight different travel speeds (Test B) revealed the ability of the developed test system to differentiate among operating conditions that may influence the performance of an auto-guidance system.

It was shown that relatively high travel speeds resulted in a substantially higher auto-guidance error as compared to the slower speeds. Using the RTK-level receiver, the difference between pass to-pass and long-term performance was found to be negligible.

The increase of up to 100% in the corresponding error estimated between Test B and Test A could have been caused by a variety of factors including differences in tractor and auto-guidance systems. However, only high travel speeds caused autoguidance errors to increase greater than 50 mm. This indicates that a standardized test procedure should include at least two specified test travel speeds (e.g., 2.5 and 5.0 m/s). Neither test showed any difference between slow and medium travel speeds.

Therefore, testing with slow travel speeds should be considered only for special circumstances. Finally, it was noted that using the outer marks did not allow sufficient space for the system to reach steady-state operation when traveling at the high speeds.

(Source – http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1172&context=biosysengfacpub)

Leave a Reply

Your email address will not be published. Required fields are marked *