Well think again!
Last night we had a big breakdown on bemyeyes.org.
Some famous actor mentioned us on facebook - which is by the way great, and well the database could not handle it.
Within 6 minutes the database went from responding within about 3-600 ms to 6-8000 ms.
I dived into the problem and it made no sense to me since I didn't deploy anything new. But it was clear that the database was being slow.
I used google and could find no sense of it all.
I thought with the publicity Be My Eyes has had in the last days, and it being a non-profit org that maybe someone would help out on the internets :-)
@BeMyEyes is close to being down at the moment. @MongoDB is very slow. it went from answering in 300 ms to 8 s. Investigating— Klaus Hebsgaard (@khebbie) January 18, 2015
And well so many people wanted to help out, I was pointed to tools to help.
Eventually Pedro from Heroku showed up and was added to our hipchat channel.
For more than an hour we ping - ponged (actually Pedro did most of the thinking, and I pasted code into the shell).
And in the end we solved the problem.
Meanwhile 4-5 other people sent me recommendations on what to do to find the problem.
So I must say that it warmed my heart to see how people on the internet really cared about Be My Eyes and wanted it to succeed.
So A big thank you! goes out to Pedro and Heroku, and the other Guys trying to help out on twitter:
- Pedro Belo
- Lucy in the sky
- Miles Ward
- Sandeep Parikh
- Andrew Homeyer