is there any reason to upload more than the legacy 2016 kismet .nettxt?

[cortijocare]

I've searched these forums and found posts dated 2016-2017 on the topic. As of 2019 however, and having gone through each field in the 2016 legacy edition of kismet, I cannot find any 'odd' field in either the gpsxml or netxml not present already in the nettxt (which evidently is much smaller in size than the other two).

However, and before i upgrade to kismet 2019 now that it has csv export support as of 4 days ago, I'd like to check if I'm right in thinking ONLY the .nettxt is need as it consist of peak/avg/bssid/name/encryption etc etc information?

Thank you

[cortijocare]

Actually, it's worse than that. I'm now getting concerned if wigle even tries to dedupe invalid ssids contained in gpsxml and netxml vs the 'clean' info contained in nettxt. If not, then it seems it would be trivial to cheat wigle by uploading junk data. IMHO gpsxml contains such junk data - see the output here:



As you can see, the gpsxml file has been 'filtered' down to single placements of anon ssids, the nettxt is working as intented but without the wigle 'magic sauce' to try and place the ssids where they physically were located and the netxml is essentially a mix of the two, anonymous ssids located at the first gps location and no other - what a mess!!!

I'm going to upgrade to kismet 2019 and try to output to csv to see if things are better ...

[cortijocare]

Update: even worse, not only wiggle ignores the signal data in the nettxt file, it doesn't even try to associate it with the matching ssids in the other two files....


Ok, at the risk of talking to myself, i took the time to upload 3 files i knew i had run a perfect square round the block and downloaded the resulting kml *from wiggle*. This is using kismet legacy 2016 on kali rolling.

.nettxt output:


Looks perfect to me but everyone here in the forums insists it's 'missing data' - please explain.


Then we have a problem:

.gpsxml output:



Would be harmless if this didn't register in wigle as 6774 new wifi location, all with blank ssids, and as far as i can tell, wigle makes no effort of matching these to the previously uploaded files, meaning this doesn't appear to improve wigle's database one ioata. Just fill it with junk. That said, it appears to be smart enough to de-dupe the nameless ssids, so technically so far it's 'only' doubling the # of ssids on file.

Things get even more puzzling with the netxml:



.. where the networks are correctly displayed, just not exactly in the same location (leading me to think it's reading from a different field than when processing the nettxt, ie. could be reading from peak instead of avg lat/lon).

In conclusion, here's the 3 files overlaid and the outcome from the wigle upload page:





And before you say 'oh but it doesn't matter because wigle dedupes these' - the answer is no, it doesn't - the final map on the wigle homepage reflects exactly the combined overlaid 3 kml files.

TLDR: people upload all 3 files because the forums tell them to but i think that's a bad idea(tm) - only the nettxt is needed when using kismet 2016 legacy. UPDATE:... but wigle.net will not process some data including missing ssid strength field.... which uploading the rest of the files will not fix... *sigh*

[arkasha]

This is a really cool experiment!

A word of warning on methodology: it appears that you're trying to infer database behavior from the KML exports per-upload, which isn't working the way you expect it to.

If you want to estimate the signal effects, I'd use the API to query distinct BSSIDs after each upload rather than the KML export tool that profiles the upload.

Cheers,

-Ark and the WiGLE team.

[cortijocare]

Thank you very much Ark. I'll upgrade to 2019 r2 and will report back based on direct ssid querying.

[arkasha]

Cool, please keep us posted here!
This is probably the endpoint you'll want to use to see updates to the trilaterated position:
https://api.wigle.net/swagger#/Network_ ... s/search_1

However in order to see the list of observations per network, check out:
https://api.wigle.net/swagger#/Network_ ... ols/detail

and of course for your API keys and sample CURL command templates check:
https://wigle.net/account

Cheers,

-a

[cortijocare]

Hi Ash, so I upgraded to Kismet latest and I'm using their CSV output option to upload to wiggle.
I'm satisfied the file contains all the information needed, effectively removing the need to upload the 3 large output files from the 2016 version.

Unfortunately, there is also what appears to be a major bug - when i try to download the file resulting KML, i get an empty (0 byte) output. I tried 'save link as', clicking it, etc... no dice. This is odd as the other uploads (direct from the wigle phone app) function just fine.




Is this a known issue? Are my new locations still being accounted for?

Thank you

[Eviltom]

Unfortunately, there is also what appears to be a major bug - when i try to download the file resulting KML, i get an empty (0 byte) output. I tried 'save link as', clicking it, etc... no dice. This is odd as the other uploads (direct from the wigle phone app) function just fine.

I came over to the forum to ask exactly this. I've just setup a RPi Zero as a wardriver after my phone updated to Android P and made the app useless.
I uploaded a small test csv generated by Kismet2018 and although processed and showing 166 wifi with GPS, it won't show let me download a KML so I can sense check it. I've uploaded a couple more files which are in the queue now so I'll see what happens.

[arkasha]

oh interesting - we'll check it out.

As a reminder, those KML files are just about your uploads, not a good way of evaluating the WiGLE trilateration tech.

[arkasha]

looking through the logs, I get:

Code: Select all

No locations results for transid 20190305-0008

- if the file doesn't contain coordinates, wigle doesn't attempt to build KML. since the link is a direct download, there isn't a good channel to provide an error message.

feel free to post transids here, I can check them out if you think you've got uploads with valid coordinates that aren't generating KML.

depending on which element of the Kismet output you're trying to look at KML for, you might have a zero-coordinate timestream, or your GPS may have failed to respond.

-ark

[cortijocare]

Ark, regarding 20190305-00008 (and for that matter all my kismet-based uploads), it's a standard csv file generated by kismet_to_wiglecsv. It's really frustrating to hear that it doesn't seem to 'work' - because inside it looks like this:

86:2A:A8:3B:54:0C,ECCI_AGORA,[WPA2-PSK-CCMP][ESS],2019-Mar-04 12:41:26,6,-92,36.475353,-4.981659,-1.555000,0,WIFI
86:2A:A8:3B:54:0C,ECCI_AGORA,[WPA2-PSK-CCMP][ESS],2019-Mar-04 12:41:26,6,-88,36.475353,-4.981659,-0.100000,0,WIFI
86:2A:A8:3B:54:0C,ECCI_AGORA,[WPA2-PSK-CCMP][ESS],2019-Mar-04 12:41:26,6,-89,36.475353,-4.981661,0.700000,0,WIFI
86:2A:A8:3B:54:0C,ECCI_AGORA,[WPA2-PSK-CCMP][ESS],2019-Mar-04 12:41:26,6,-87,36.475349,-4.981658,0.800000,0,WIFI
86:2A:A8:3B:54:0C,ECCI_AGORA,[WPA2-PSK-CCMP][ESS],2019-Mar-04 12:41:26,6,-90,36.475349,-4.981658,0.800000,0,WIFI

.. and so on. In other words it appears valid unless I'm missing something obvious.

I'd really appreciate if you could have a look - I've build this rig just for the sake of wigle and i often walk around town just for the sake of wigle... I was planning a 10x antenna box as the next project to scan even faster.

The transactions ID are:

20190309-00557
20190309-00554
20190308-00899
20190305-00008
and...
20190304-00102

Thank you so much for your help.

[arkasha]

ah, I've found the problem - the time value we populate in that table isn't getting set correctly on the observations. watch this space for details.

[arkasha]

ah, kismet is inserting "Mar" instead of "03" in the date stamps, which doesn't match our format.

[arkasha]

documented issue here:
https://github.com/kismetwireless/kismet/issues/122


proposed a patch here:
https://github.com/kismetwireless/kismet/pull/123

happy to hear if i've made a mistake - it's been a long time since i've used

Code: Select all

std::put_time

[cortijocare]

Thank you very much Ark, wouldn't have noticed without you. Quick question: is there a point for me to continue wardriving until the patch is in? I wouldn't want to collect endless points only to not see them appear on the map... thank you!