Seti is down again

[Richard Haselgrove]

Now the BOINC website is down as well - unsurprisingly, since they live in the same building, with the same power supply. It was pre-announced.

[corsair]

Now looks like that BOINC is up or at least the web page is working, but no DW/UP in WU neither crunched results for SETI.

[msattler]

Now looks like that BOINC is up or at least the web page is working, but no DW/UP in WU neither crunched results for SETI.

 

Well, that should mean that power has been restored.
Now we just have to await the arrival of the boyz in the lab tomorrow morning to  set things alight.

[Cosmic_Ocean]

There was a blip of activity on the Crickets.  Looks like it's all down again though after only ~15 minutes.  I'll fight for upload slots in 12+ hours.

[Richard Haselgrove]

Uploads are starting to happen, but a bit slow and hit-and-miss.

[perryjay]

Missed it by that.......much.  Finished my last work unit less than an hour before they came back up. Reported 272, requested new work and got one whole shorty. Took me all of four minutes to complete and report. Of course then I got no work available for a try or two but now have around twenty work units to crunch on until they get this mess cleared up.  Could still use some for my CPUs but at least my GPU has something to do.



Well, while I was typing they were nice enough to send me another twenty tasks. All resend lost tasks. Didn't even know they were missing. 

[msattler]

http://www.youtube.com/watch?v=oPwrodxghrw

[Cosmic_Ocean]

My server comms were mundane at best.  Allowed network comms, all uploads went through on the first try, all reported on the following work request.  AP-only machine got no new work in response, MB-only machine got a full cache (2.5 days, ~15 WUs) of _0's in one request.

[msattler]

Oops...
Looks like Seti went dark about 2 hours ago.....forums and all at the moment.
Home page slooooooooowly loads with the following message....
Unable to connect to database - please try again laterError: 2003Can't connect to MySQL server on 'carolyn' (111)
I thought they had dumped MySQL........

Meouch.

[[seti.international] Philip J. Fry]

Project is temporarily shut down for maintenance

At least one is working ..

[[seti.international] Philip J. Fry]

Still no connection possible ..

Scheduler request failed: Couldn't connect to server   
Scheduler request failed: HTTP internal server error   
Scheduler request failed: Timeout was reached

[[seti.international] Philip J. Fry]

OK, S@h is again reachable .. - at least for my machine ..

[Richard Haselgrove]

OK, S@h is again reachable .. - at least for my machine ..

Front page news, 22:56 UTC last night:

We are recovering from a database crash
The main boinc database suffered a crash this morning. We are back up and running now and are catching up on work distribution.

[msattler]

Richard....
Just curious though we are back up and running......
I thought I read that they had converted away from MySQL.
Was that only on some of the servers?

[Richard Haselgrove]

Richard....
Just curious though we are back up and running......
I thought I read that they had converted away from MySQL.
Was that only on some of the servers?

They've always run two database systems - MySQL for BOINC (WUs/tasks/message boards), and IIRC Informix for the science databases and NTPCKR. So far as I know, there's been no change there - it would be a big job.

You might be thinking of the recent change from Apache to nginx - that's the webserver that handles comms (including uploads and downloads), not the database.

If there is a worse time for something to go wrong, it will happen then.

- Murphy's Law