First, I would like to thank everyone for their patience during the last 2 weeks and sincerely apologize for the downtime. We received thousands of inquiries during this time as to the state of the forums. While we were unable to reply to every email, we were working diligently at getting the site back up.

>The Problem

The events that occurred were a series of unfortunate circumstances. In brief, the database encountered a serious corruption that was not recoverable. Although backups take place daily on the site, the backups themselves had been failing. Unfortunately, the ISP who is accountable for the backups, was unaware of these failures. Furthermore, the failed backups were over a period of time where the retention cycle for the tapes had reset leaving no tape containing any usable data. In addition to these automated backups, manual dumps of the database take place on the server monthly. Due to the size of the database and existing corruption, some of these dumps were also unusable. To make matters worse, the entire high level staff at MBWorld that would have addressed these issues (including myself) were out of town on business.

>The Fix

By taking pieces of the database from various usable dumps, we were able to restore most of the actual database up to September 15, 2003. Regrettably, this has left 6 weeks of content unaccounted for. Unfortunately, there was no way to restore this missing content due to the lack of a usable restore from tape. Any new user joining MBWorld during these 6 weeks would need to re-register.

Note: Your Default Thread View may only allow you to see content for the past 30 days, so you may not see any posts in some of the forums. This will change as new posts are added. To change your Default Thread View to see posts older than 30 days, select 'user cp' from the upper menu, then 'Edit Options' and change your Default Thread View accordingly.

>Steps To Mitigate Such An Event In The Future

Certainly this has been a learning experience for all of us here. Our staff has been working day and night to get the site back up as quickly as possible. Several components have been implemented to minimize another such incident:

1) Automated daily backups are now verified and checked
2) Manual backups take place on a weekly basis (these are only used if option 1 fails) and sent offsite
3) Rigorous maintenance schedule has been implemented to ensure any database corruption is identified and addressed
4) Application components have been patched to address several existing bugs
5) Senior staff will be much more readily available to address such an issue in a timely manner
6) New forum components and functionality will be added to assist in the administration and maintenance of the forums
7) A new server will be brought online in the next few weeks boasting faster hardware


All of us here at MBWorld hope you continue to make MBWorld your home. We ask that you inform your fellow colleagues that the site is back online! Please feel free to email me directly with any questions or concerns at mister_vanos@mbworld.org.

Sincerely,

Simon A. ("Mr. Vanos")
MBWorld Administrator

 

I think I speak for most of us when I say I am just glad everything is working now. I know a lot of us who had temporarily migrated over to one of the other discussion forums weren't sure if the sites were ever coming back up. I truly appreciate all of the effort you all put in to making these forums more reliable as the information we get here is invaluable.

Regards.

 

Thanks for the support and encouragement Brian! It's for people like you that I didn't mind sleeping a total 5 hours this last week

 

What kind of database are you using?

Erik

 

MySQL

 

Mr V,

I just wanted to say thanks to you and everyone else who was involved in bringing the forum back up. MBWORLD is a great community that was sorely missed while it was down.

David.

 

Thank you David!

 

wow, that's one hell of a f*ck up!!

After 5-6 days i kinda figured that it was a BIG problem, you did incredible work and we are all happy and impressed! Keep up the good work! Glad it's back!

 

Mr V.
Anyway I can get back my PM stuff. I was in the middle of a buying some parts and lost all contact information with the seller. My original name was "benzoben". I had to register and used my new name. Please PM me if possible.