The GPS debate

Select to expand quote

boardsurfr said..
Instead, let me address the claim that the accuracy of approved GPS devices was "scientifically evaluated". Sorry, mate, but that is a false claim. "Evaluated by scientists", maybe; "evaluated with scientific methods", maybe. But there are some hard, indisputable requirements for science, and they include publication of methods and results in a manner that allows reproduction by other interested scientists. Tom Chalko has at least made some attempts back in the GT-11/31 days, but there is pretty much nothing available for the GW-52. If you cannot take the time to publish your methods and results, don't call it science. A close-knit group of guys making decisions that affect a lot of others? That's called politics, not science.

I didn't know what to say in response to this when I first read it.

Do you really believe that the technological enhancement of GPS's, *shouldn't* be made by a group of scientists (mathematicians, engineers, programmers) ? ... who else is in a better position to advance the technology for the betterment of the sport ?

I will never leave GPSTC and I'm sure we will eventually get more gps devices approved or new suitable ones will come on the market. I also like the local groups, so why we cannot play on both and be friends?

etc....

Select to expand quote

mathew said..

boardsurfr said..
Instead, let me address the claim that the accuracy of approved GPS devices was "scientifically evaluated". Sorry, mate, but that is a false claim. "Evaluated by scientists", maybe; "evaluated with scientific methods", maybe. But there are some hard, indisputable requirements for science, and they include publication of methods and results in a manner that allows reproduction by other interested scientists. Tom Chalko has at least made some attempts back in the GT-11/31 days, but there is pretty much nothing available for the GW-52. If you cannot take the time to publish your methods and results, don't call it science. A close-knit group of guys making decisions that affect a lot of others? That's called politics, not science.

I didn't know what to say in response to this when I first read it.

Do you really believe that the technological enhancement of GPS's, *shouldn't* be made by a group of scientists (mathematicians, engineers, programmers) ? ... who else is in a better position to advance the technology for the betterment of the sport ?

You missed the point, so I highlighted it in bold. Science requires that you publish your results and methods, with sufficient details so that others can try to reproduce your results, or apply your methods to other problems (or, in our case, GPS units).

Since the methods and results of evaluating new GPS units have generally not been made public at all, anyone who wants to see how accurate his new, non-approved GPS unit is pretty much has to re-invent the wheel. Simply saying "it must have accuracy estimates" is not the answer, since (a) accuracy estimates have to be verified the same way as speed estimates, and (b) speed estimates can be verified even in the absence of SDoP or similar values.

I'd love to see alternative GPS units being scientifically evaluated. Of the groups you list that have been involved in GPS evaluation so far, however, two are (usually) not scientists: engineers and programmers. Sure, we'd want engineers and programmers to be involved in the process. But these two groups have nothing similar to a "scientific publication requirement". Quite the opposite, they usually deal with stuff that should not be published (trade secrets and copyrighted material). As a trained scientist who works develops software, I am quite familiar with these differences.

What would be useful would be a protocol about how to evaluate the accuracy of a GPS; call it a SOP, if you want to. It would probably involve multiple tests of dual units, comparing them to previously approved devices, as well as criteria about what is and what is not "acceptable accuracy". This would make it easy to reject any GPS devices that are not up to snuff. It would also enable others to do some of the work that is currently done by a very small group of scientists, mathematicians, engineers, and programmers. In my eyes, that would be a much better situation than frustrating somebody new every few months, as it happens now.

Select to expand quote

John340 said..

boardsurfr said..

What would be useful would be a protocol about how to evaluate the accuracy of a GPS; call it a SOP, if you want to. It would probably involve multiple tests of dual units, comparing them to previously approved devices, as well as criteria about what is and what is not "acceptable accuracy". This would make it easy to reject any GPS devices that are not up to snuff. It would also enable others to do some of the work that is currently done by a very small group of scientists, mathematicians, engineers, and programmers. In my eyes, that would be a much better situation than frustrating somebody new every few months, as it happens now.

The following info from the FAQ tab on GPS-Speedsurfing web site provides the SOP

www.gps-speedsurfing.com/default.aspx?mnu=item&item=gpsother

That is a very interesting page. Note that it states:
"In general we allow any GPS Devices that’s is capable of logging GPS Data with a sample rate of 1 per second and support the .GPX data format". At another place, GPS-Speedsurfing.com explicitly allows GPSLogit, without restrictions on which phones can be used.

It then gives the hardware and firmware specifications desired in new "record approved" devices. It does not, however, give any information about how devices are tested, except saying that two devices should be sent to them, and that they will be tested by an "independent test authority".

This is a pretty sensible approach for GPS3. For the GPSTC competition, it's a bit too relaxed, since it allows posting from units that are known to be inaccurate.

Select to expand quote

Simon100 said..

So lets get this bit stright to the sdop values are a margin of error does any one have the actual formula or method used?

The only description I recall is that the SDoP values in the Sirf chips is based on the co-variance of the Kalman filter. The ublox chips give a similar value with a different name which is calculated in a somewhat similar way.
What really matters, though, is to have a good idea what the relation between actual errors and accuracy estimates. With good descriptive statistics, we can decide how to use the values properly in the data analysis.

Select to expand quote

Simon100 said..

Cant we just use a combination of the doppler and heading data to get a better aproximation, what are really the odds that the doppler will read unusually high in your direction of travel right when your doing that 2 second run ?
From everything i have observed errors come from changing angles and bumpy water not just random gps spikes.

It's definitely worth-while to look into other ways to estimate accuracy, too; # of satellites and acceleration are already used to filter out bad data points, and often work well enough for really bad spikes. But you're looking at pretty heavily processed data, and results will vary by GPS chip, device, and even chip settings.

Select to expand quote

Simon100 said..

How many examples is there of large errors in the gps readings? the only ones i have really ever seen were actually the ka72 software which has now been fixed.
If i actually put 2 of each gps on a car straight above a wheel that is reading wheel speed off an abs sensor ( above so there is no errors in corners) will people accept that as true indications of the acuracey of each ?

I have seen plenty of relatively large spikes in GT-31 data, and (I think) a few in GW-52 data. The biggest I have ever seen was a > 1000 knot speed in data from a different chip (that apparently does not use doppler speed, but is quite popular otherwise). Could have been a problem specific to the unit I had, though, which I returned to the manufacturer.

The ABS sensor idea sound intriguing, but you'd have to verify that it actually is more accurate than our typical GPS units (and probably adjust for things like tire thread etc). We are talking about single-digit percentages or less; I'm not sure if ABS sensors need to have that level of accuracy, but I don't know anything about them. I once looked into using the accelerometer in phones for this purpose, but these things are way to inaccurate; they mostly need to know where up and down is, and if you are shaking the phone.

I think driving data can be a useful first step in the evaluation. It's very easy to gather data from multiple devices that way. Just avoid the bumpy roads :-). In paired GW-52 driving tests, I saw a nice correlation between the accuracy estimates and how closely the spikes resembled each other: spikes were in sync at low SDoP values, but out of sync when SDoP increased. No big surprise, but still nice to see. Since I was driving a big van on relatively smooth roads, I did not expect to see as many spikes as I did. I'm planning to repeat this with a few additional GPS units that use different chips and electronics, but it has been too windy recently to put these together.

Select to expand quote

Simon100 said..

So to really simplify it the sdop does nothing but compare the speed reading to the expected speed which was only calculated with averages ?

I think that's a bit too simple. I don't think anyone outside of the chip companies actually knows how exactly SDoP is calculated, and the accuracy estimates from the ublox chips are most likely calculated differently than the Sirf SDoP data. The CPUs in the GPS chips do a lot of work to calculate doppler speed from the raw satellite data, effectively reducing many numbers to a single number. That gives you lots of data for a variance calculation.
The approach you describe is something that could be done even with the final numbers - using acceleration goes into this direction, and can be useful.

I'd love to see some ABS sensor data, but I'd have no clue how to get to them, nor do I want to fiddle with the electronics in my car. Checking ABS sensor data for the synchronized spikes that can be seen in dual GPS data would be a good way of differentiating between real speed differences and systematic artifacts.

Select to expand quote

decrepit said..

Simon100 said..

>>>>
If i actually put 2 of each gps on a car straight above a wheel that is reading wheel speed off an abs sensor ( above so there is no errors in corners) will people accept that as true indications of the acuracey of each ?

Not sure if everybody would accept it as an absolute accuracy check, but I think it would be well worth doing! As boardsurfer says it could shine some light on what the higher Hz gps ripple is, and may give and indication about SDoP, especially if you can arrange a varying satellite shade over the gps.

Yes agreed it would not be the best on absolute acuracey infact i think the best way to calibrate it would be against the gps and any errors in the wheel speed measurment should very predictable( like it might get more low at speed) and not irratic like the gps it is consistant enought that is converted to the rate of acceleration a cylinder missing is clearly visable in the data ( actually that was from a tail shaft speed sensor which is far less acurate butalso has axle flex to spring back ) . But even of the plotted lines from the gps and wheel speed seperate in a even manner this is not a concern as what we are really looking for is the fluctuations in speed from the gps. the behavior in corners and under deceleration and acceleration and of course as mentioned when its view of the sky is partially blocked. Idealy i guess some meat from the butcher should be used to simulate that but its to messy for me in that test i bet the new one with the helix antenna wins easily .

Select to expand quote

Simon100 said..
>>>

Yes agreed it would not be the best on absolute acuracey infact i think the best way to calibrate it would be against the gps and any errors in the wheel speed measurment should very predictable( like it might get more low at speed) and not irratic like the gps it is consistant enought that is converted to the rate of acceleration a cylinder missing is clearly visable in the data ( actually that was from a tail shaft speed sensor which is far less acurate butalso has axle flex to spring back ) . But even of the plotted lines from the gps and wheel speed seperate in a even manner this is not a concern as what we are really looking for is the fluctuations in speed from the gps. the behavior in corners and under deceleration and acceleration and of course as mentioned when its view of the sky is partially blocked. Idealy i guess some meat from the butcher should be used to simulate that but its to messy for me in that test i bet the new one with the helix antenna wins easily .

So are you up for giving it a go, sounds like you know how to latch on to the ABS signal? I'd love to see the results.

only if you wear it on your head !

Wow. Pretty much what we all wanted. I know the under /over grip affects accuracy but I would probably strap it to boom near the boom clamp anyway as being old and presbyopic I have difficulty seeing small things close.
The 50m depth , million data points and 1cm/s SDOP are nice touches.

Don't get carried away yet. It still needs to be tested to see if it lives up to their claims.. This is the first I have heard of it and it may still suffer from the micro antenna issue. I wonder why they have not actually consulted with those who have worked with them and bought from them in the past about this development?

Can they use an older photo for windsurfing ? Or do they think it represents the targeted audience ??

"That's awsome......comes with built in vibrator"
Piv,what were you thinking of strapping it to?

Hi all
there seems to be two separate trains of thought going on here from what I can see. I understand that a number of conversations have been had in recent months regarding the merits of certain gps units etc.
I think something is frustrating people is the fact that the gpstc technology committee seem to have a preconception based on years of being involved and time spent volunteering their time that is greatly appreciated by all. This is the fact that we require a particular device and a standard to be able to list results on gpstc. There is definately merit in having accuracy and standardised results that are able to be compared and tested.
I for one find it confusing when we don't have clear rules around what can and can't be used.
I think what sailquick has stated above is that currently there is only one new device that we should be using that meets the standards of the committee. What i'm a little unclear about is what the standards are other that it needs to have verifiable data.
I know that all of the data that Ian (Cocky2) has posted is freely available as stated on ka72 for any to download and assess. There seems to be a resistance to even considering this information based on the above mentioned preconception even if it is based on experience/facts.
The watches in question have their own app that give great on water feedback and personally from what I have seen although not that cheap are far easier than the login app which is also great.

I for one as a person that does tend to drop a lot of gybes (still learning) finds the idea of buying and using a gw52 a problem due to the fact that even with an aqua pack and another sealed bag inside that I have had my device get a little damp a few times.
I would much prefer the idea of a watch that can get wet for $450 than a device that may get wet and die for say $250. Also if that is what is required to post we all have to make our own minds as to weather we purchase one and continue to be involved or not. It is an expense but in the grand scheme of things it is not as expensive as the latest sail, board or boom that we all seem to find the money for.

I for one think that we require some leadership from the committee regarding the whole situation in that we should have some clear rules as to what device you can post with. Moving forward however perhaps there is the possibility of a debate/forum to discuss the direction that the gpstc takes in regards to device's. It needs to be clear based on what team members want and then there could perhaps be a vote taken similar to any other club meeting that would confirm the direction that the competition takes in regards to the ongoing use of certain devices and the requirements that use for gpstc. Clearly this would require a great amount of time and effort from the committee and also the ability to accept to decision of the members whatever it may be.
I Believe that when looking at this discussion and many others recently it has become a real sticking point for a number of people involved.
Good luck to all involved
Let hope the summer winds are awesome.

Select to expand quote

Bugs74 said..

...Lots!

There are no preconceived ideas, and no ideas/decisions that are not based on technical merit, a huge depth of experience, the advice of very technically knowledgeable experts and appropriate scientific testing.

The rules are very clear: GT-31, GW-52 are the gold standard and must be used for posting scores that count for the team or for PB's and the top rankings.

The Canmore was allowed as a gap filler when nothing else was available and has since proven to be highly problematic. Although it was done in good faith, for what we thought at the time were good reasons, in hindsight was a mistake. No one who is serious about their PB's or rankings should be using them now.

Ian's data is next to useless for any serious, meaningful comparisons, as, at the very least, there are no controls. Multiple units would have to be tested side by side with controls, in controlled positions and situations for any useful data. All it does is confuse and muddy the waters.

There is no doubt that the watches and GPS-Logit are extremely useful for on the water feedback.

Again: The leaderships is here, the rules are very clear (like them or not), and there is no 'Debate' if we want a fair competition that is not based on fantasy and chance. We all have our wish lists for what ease of use and other features we would 'like', but the reality is, many of those things are not available now, and may not be in the foreseeable future.

That's it from me in this thread. Enjoy your sailing.