WiGLE.net

Hi every one!

I'm a Phd Student at George Mason University and I'm trying to do a research project on WiFi encryption but I need to figure out a way to download chunks of the data in a way that preserves the latitude and longitude coordinates as well as the network names and encryption status. Does WiGLE have a way to download the data in any format other than geo/meta? Is there a way to convert geo/meta data into gpx or csv?

Thanks!

Same problem here

Hi, yes very good point. In fact how do I download wifi location points for the UK?

Check out our JiGLE java client, it may help you on your pursuits.

Bobzilla,
Much obliged! I've tried the client today but after specifying several sizes of area in the UK (via modifying the map pack file lat/longs) jigle/ digle only seems to return 10000 results per query! Is this a limitation of the software or am I doing something wrong?

Kind regards,
Tom

There are technical limits to prevent abuse. Repeated queries will populate more, up to a point.

There are technical limits to prevent abuse. Repeated queries will populate more, up to a point.

What sort of abuse?

I came over to the forums to ask a very similar question about the data (CSV, whatever, give out a real format, or point to ways of parsing what you do give out) - I read that nothing is thrown away, but you can get only fragments of it.

You also - apparently - can't get historical data.

Can you point to a document that says what is or isn't allowed?

And if legitimate researchers can't have the data, who can?

JiGLE is an open source java client available on the Downloads Page, which with little effort you can get CSV data from. Not sure where the "can't get historical data" idea comes from, you will likely get that first in a dense area. There are some (rather high) limits to prevent continuous beating on the WiGLE servers, which a researcher is unlikely to hit.

JiGLE is an open source java client available on the Downloads Page, which with little effort you can get CSV data from. Not sure where the "can't get historical data" idea comes from, you will likely get that first in a dense area. There are some (rather high) limits to prevent continuous beating on the WiGLE servers, which a researcher is unlikely to hit.

If JiGLE can do this, great; can you point out where it can, I'm happy to RTFM? Feedback from a user trying to do just what you say - there is no apparent documentation that I can find for JiGLE other than the README.txt file, which doesn't mention anything about exporting or saving data to any formats, CSV or otherwise. The Menu options within the program don't appear to offer anything - file authenticates or connects, view... well, views, filter changes the display, preferences is blank and stats brings a popop with an overview. Help brings up a version number. Searching google for "JiGLE documentation", export, CSV and the like appears to show a lot about how to use it to import data. Is there a page or something that describes any of this?

Can't get historical data. OK - I live in seattle, WA. I type in WA into the search engine, and the 3rd entry that comes up is 06:24:36:AC:DC:15. The search tool tells me that the first time it's seen this MAC/ was 2010-06-13 22:12:01, and the last time was 2011-02-20 18:12:29. It then proceeds to list various bits of data about the network. But without a timestamp on the data, how am I supposed to know when the fields were populated, or if they've changed? For instance, take the channel - according to this, it's 8. Was this from 2010/6/13, 2011/2/20, or something inbetween? Has the channel changed during that time? Was it seen in every import of data in this area? What HW/SW tools were used to generate the data - different scanners have different behaviors.

The data is potentially incomplete, too - for instance on an iphone is has a "Confidence" field - is this in there anywhere, or has it been stripped and discarded?

It appears that an enormous amount of effort has been put into displaying the uploaded data to a certain type of user, and I applaud that. But I read that > 30m networks have been uploaded to your database, and I'd like to see the rest of it. Database size-wise that's a very small amount of data, and I could easily consume the entire amount - your comment about being unlikely to hit the limits is pretty laughable, I want more than what you have, not less.

I can appreciate concerns for hammering your systems being taxed or possible bandwidth issues or anything else. But you appear to have some useful data somewhere, and there are people out here who would like to use it in ways that you might not anticipate or think are useful. If you don't care to share it, that's your prerogative, but if not it'd be nice to have a policy or document that states such a thing. You might consider asking if others would like to mirror the data as well, that might relieve some of the stress on your own systems.

p.s. Your method of grabbing consolidated.db will not work on all macs. I'll write something about this in a separate post. You might consider having some additional forum categories, as it's not really a project suggestion bit... maybe have a tech forum place?

What I did was this...
Installed jigle
Copied and modified .mappack file in notepad or simple text editor
- changed short name
- changed ref coordinates to area of interest
Saved and closed mappack
Started jigle
Chose new map pack at top left of the window
Authenticate Login in prompt to draw down data from wigle
When complete I searched for the latest modified .auto cache file in the data directory
I copied it and renamed the extension .txt
Opened it in text editor
Ran find and replace - find all tildes and replaced with commas
Saved file and renamed extension .csv (voila!)
The last two steps can be skipped if you have an application that can import a text file with a tilde as a separating value (ms access in my case)

From what I can see a few attributes are stripped (accuracy value and encryption type being the most notable), but the date of detection is available. If you want to give it a try or want me to do a test extract for you then I can give it a go if you can give me a few streets or a zip code to use for the extract?

Regarding the limitations, I suppose it is a bit annoying to be restricted on the number of the records but i understand the reasons given why this is the case.

I can appreciate concerns for hammering your systems being taxed or possible bandwidth issues or anything else. But you appear to have some useful data somewhere, and there are people out here who would like to use it in ways that you might not anticipate or think are useful. If you don't care to share it, that's your prerogative, but if not it'd be nice to have a policy or document that states such a thing. You might consider asking if others would like to mirror the data as well, that might relieve some of the stress on your own systems.

your nascent enthusiasm is appreciated.

there is an ongoing research project amongst the kittens whose mewling powers the wigle engine; every once in a while (usually out of their eyeballs on catnip) they grab their lynch and lamport off the shelves and start hammering away at a distributed, cloud-enabled, data noise management engine without all of the pesky, boring reliabilty that has been such a feature of the last nearly-a-decade around these parts. they inevitably come up against the coordination costs, consistency/decidability issues, and the fact that every other single piece of the intertubes is as unreliable as their food supply, and go back to making things just work.

one of the interesting side effects of making things work is that the kittens' idea of what is easy/doable/a good idea doesn't always jibe with the rest of the world, who get to sleep peacefully at night, unaware of the horrors that lie within.

these bouts of eigenmadness also tend to eat up the time of the committee for committee formation whose original charter was to devise a charter to produce policy documents, so they tend to just run with what makes sense. to them. at the time.

that being said, TomThumbUK's approach is what i myself do, when i happen to need some of that kind of data.