WiGLE currently has 372 files queued for processing

[NetBandit]

Someone might want to check on the queue.

WiGLE currently has 372 files queued for processing

as of 20060824 @ 8am, PDT

thx
-NB

[KH]

WiGLE currently has 372 files queued for processing

I see items get removed from the queue, it's just verrrry slow so there is a buildup of entries.

The dual-hamster setup for keeping the site running must have one hamster sick or very tired. Please, think of the hamsters!

[Dutch]

WiGLE currently has 372 files queued for processing

I see items get removed from the queue, it's just verrrry slow so there is a buildup of entries.

The dual-hamster setup for keeping the site running must have one hamster sick or very tired. Please, think of the hamsters!

No no no.. It's the interns.. Or rather the new intern. Her name is Monica and she is busy with the server admin. You'll see the stained dress on the front page when the investigation starts.

Dutch

[whitedice]

67 files parsed. 0 new networks today. After 12 hours?

[mahlerrd]

I wonder, as a curmudgeon who doesn't like to attribute to coincidence anything that could be explained in another manner, if the intern's new "garbage-finding," ranking and triangulating algorithms might need a bit more testing?

Or, as is usually the case when I write something, is there something in the dataset that is confusing their algorithms?

[ax0n]

67 files parsed. 0 new networks today. After 12 hours?

With the new algorithms, it is highly possible that there are hundreds of "cleaned up" datasets that are being re-processed which contain old data, therefore, no new networks, just better data for those networks.

[whitedice]

Does that mean the triangulation portion of the parsing is what is taking so long as it reconfigures where existing detected networks are (in so many words) repositioned with better data?

[mahlerrd]

Hey, you guys could create a queue of delayed items and a watchdog process on the regular queue - whenever the watchdog process notices there's no items waiting to be processed, it could inject one of the delayed items.

Then, punch a bunch of those backlog jobs into the delayed queue and off you go.

Of course, the presumptions and assumptions are many in the above, so if it bears great similarities to the ramblings of a lunatic, it's because...

Oh wait, I don't think I ought to complete that thought.

[ax0n]

Well, I haven't seen the WiGLE Queue get actually clogged to the point of no return since 2004 or so. Since this massive data dump is probably a one-time thing, it probably doesn't make sense to burn a lot of effort on making a high-priority and low priority queue system.

I've got 3 or 4 in line ready to process as well, but points are points no matter if they take 3 seconds or 3 days to register with the system.

Patience, grasshopper.

[NetBandit]

I guess now would be a good time to remind people that they can combine their stumbles into a single upload file (merge files in netstumbler) so that the processing time can be minimized.

-NB

[NetBandit]

OK, this pretty much explains it:

1 hratch 581,795 (8.336%) 904,806 52,897 17-Jul-2002 24-Aug-2006

11 whitedice 63,137 (0.904%) 87,872 14,624 30-Apr-2006 22-Aug-2006


-NB

[uhtu]

heh. you folks are hilarious.

an extremely large ammount of data hit the site at the same time, which causes backups, which cause backups, which cause recursion :-)

the servers are up, and running as fast as their little hearts (and the hamsters-on-wheels that power them) can go. please be aware that because of some of the mechanics of the updates processing and reporting, what's *actually* been done, and what you see on the stats page will diverge until the job is done.

rest assured that we will bring the might and terror of our fully operational PokingSticks^tm down upon both the interns and the hamsters, until all the files are processed.

no byte left uncollated.

in the interim: patience, young padwans.

[Dutch]

rest assured that we will bring the might and terror of our fully operational PokingSticks^tm down upon both the interns and the hamsters, until all the files are processed.

So you're saying Monica is into the PokingStick^sm scene ?? That brings a whole new angle to the Monicagate scandal

Dutch

[theprez98]

rest assured that we will bring the might and terror of our fully operational PokingSticks^tm down upon both the interns and the hamsters, until all the files are processed.

So you're saying Monica is into the PokingStick^sm scene ?? That brings a whole new angle to the Monicagate scandal

Dutch

Did you break WiGLE again? I finally get my files uploaded, and now they're stacking up...

[y2k98]

it has been stuck at 20 all night, is everyhting ok?