Sticky Unable to access site...now problems
dalex
Posted 2019-04-25 11:19 AM (#20786)
Subject: Unable to access site...now problems



Veteran

Posts: 271
100
Location: behind the 4th wall

I have been unable to access the site for several days.  (I kept getting a message that the server was not responding.)  I was finally able to log in today but a lot of my information is incorrect, almost like the pages were reset to like a couple months ago.  Books are not marked read, reviews are missing, books I removed from my reading list are back on the list, etc. 

Is this a known issue that is being addressed or do I need to try to fix everything?

Top of the page Bottom of the page
daxxh
Posted 2019-04-25 11:49 AM (#20787 - in reply to #20786)
Subject: Re: Unable to access site...now problems



Extreme Veteran

Posts: 534
500
Location: Great Lakes, USA
I have the same problem as dalex.
Top of the page Bottom of the page
Administrator
Posted 2019-04-25 12:05 PM (#20788 - in reply to #20787)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas

Our host suffered a malware attack and had to shut down all their servers and move all clients to new machines etc. They used a backup of those sites and data to avoid moving the malware to the new machines.

Not sure what the next steps are but we're looking into it. For now I'd say stand by before you starts re-entering your data. We may be able to update the DB from our own backup. Chris is researching. Unfortunately there's not a ton we can do while we're at work so the solution may have to wait until we get home this evening.

More news as we have something to share. Thanks for your patience as we try to get back on our feet.

Top of the page Bottom of the page
Administrator
Posted 2019-04-26 1:14 PM (#20789 - in reply to #20786)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas
So the site is back up again. Let me know what issues you find. I'm sure we're not out of the woods yet.
Top of the page Bottom of the page
dalex
Posted 2019-04-26 1:20 PM (#20790 - in reply to #20789)
Subject: Re: Unable to access site...now problems



Veteran

Posts: 271
100
Location: behind the 4th wall
Everything is still wrong as it was yesterday, like it reset to a couple months ago. Books are not marked read, reviews are missing, books I removed from my reading list are back on the list, etc.
Top of the page Bottom of the page
Administrator
Posted 2019-04-26 2:54 PM (#20792 - in reply to #20790)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas
Yeah, it's a backup from mid March apparently. We will try to get a newer backup installed but it will take a while to get anything done as they are of course slammed with requests at the moment. Fingers crossed.
Top of the page Bottom of the page
Naomi_uk72
Posted 2019-04-26 3:10 PM (#20793 - in reply to #20792)
Subject: Re: Unable to access site...now problems



Veteran

Posts: 116
100
Location: Sheffield, UK
Comparing WWEnd with Goodreads it looks to me like I'm missing about two months of reading/reviewing progress - my profile here shows me still reading The Gutter Prayer which I finished and reviewed on 23 February. Have the hosting company given any indication of when/if the full restore will be completed?
Top of the page Bottom of the page
GiantPanda
Posted 2019-04-26 3:16 PM (#20794 - in reply to #20786)
Subject: Re: Unable to access site...now problems



Member

Posts: 14
0
Agree with Naomi_uk72 as to the amount of missing data.
The last book on my page "Books I've read" was on Feb 21. The next would have been Feb 24.
Top of the page Bottom of the page
Administrator
Posted 2019-04-26 3:26 PM (#20795 - in reply to #20793)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas
Not a lot of information coming from them at the moment. They’re still restoring other clients apparently. We’ve been trying to get answers but it’s just going to take time and patience. 
Top of the page Bottom of the page
Administrator
Posted 2019-04-26 4:20 PM (#20796 - in reply to #20795)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas

A2 is apparently restoring service in phases so we're crossing our fingers that they'll come back round to us and we'll get more of our data back.

As of now I suggest we stand by for a few days to see what happens before we attempt to restore anything that could get overwritten with a more recent backup.

Top of the page Bottom of the page
Administrator
Posted 2019-04-27 11:38 AM (#20798 - in reply to #20796)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas

At this time I’m not sure what’s going to happen but here is what we hope to be the case with the very limited info we have. 


The site has gone up and down several times since it first crashed so clearly they’re still struggling to fix things. We’ve seen data from 2 different backups at least so we know they have some options as to how far back they can go. This current version is from the third week of February. 


We know from what they’ve told us that they’re trying to bring their machines and clients back up in stages. We are hoping that they’ve picked a known good date before anyone’s data was compromised as the first reset and that they will come around in the next stage to update their clients to the most recent backup that is safe on a case by case basis.


We have had no indication that our site/data was actually a victim of the attack. It seems more likely that we got taken down, along with most everyone else, to stop the spread of the malware. If that is the case we shouldn’t lose a lot of data though I suspect we’ll lose at least a week’s worth at least since their backups are weekly. 


There’s a lot of speculation in that but that’s the best we have at the moment. A2’s not doing great in the communications department so all we can do is stand by while they sort things out.


I’m trying my best to be optimistic right now because I don’t like to suffer the worst case scenario until it becomes a fact. That is getting harder and harder as this goes on. I keep finding new ways in which we’re going to be royally screwed. It’s very frustrating and please believe me when I tell you that we feel your pain.


For right now we’re going to try and be patient as things work themselves out. We are trying to get more info and will pass along anything we can as soon as we can. Keep the faith friends. We’ll get through it one way or another.

Top of the page Bottom of the page
illegible_scribble
Posted 2019-04-27 11:56 AM (#20800 - in reply to #20798)
Subject: Re: Unable to access site...now problems



Uber User

Posts: 1057
1000
I've managed to recover all of the Book Request posts and Uber User posts from my deleted e-mail notifications going back to the beginning of March. 

Whatever we end up with, we'll be able to get things back to where we were as much as possible. My good thoughts go out to the Admin group as they deal with this sad and frustrating situation.
 


Edited by illegible_scribble 2019-04-27 11:59 AM
Top of the page Bottom of the page
Rhondak101
Posted 2019-04-27 2:07 PM (#20802 - in reply to #20786)
Subject: Re: Unable to access site...now problems



Uber User

Posts: 770
500
Location: SC, USA
Dave, Everyone,
I know waiting is the hardest part. Yay for Illegible_scribble's email. Once we know where we are, maybe Illegible_scribble can build a google doc with the missing info and we can crowd source the reentry. I am fairly certain that I still have the covers and author photos for my recent entries for the Crawford Award and some user requests, so those will not be as time consuming to reenter. Hang in there.
Rhonda
Top of the page Bottom of the page
Weesam
Posted 2019-04-27 5:26 PM (#20803 - in reply to #20802)
Subject: Re: Unable to access site...now problems



Uber User

Posts: 593
500
Location: New Zealand
I'm just happy I have finally been able to get onto the site today. I missed it so much. Happy to help with re-entry of whatever is needed when the time comes, and happy to wait until you give the word before doing anything else. I feel for you guys, Dave. Hope you hadn't done too much programming work that will be lost.
Top of the page Bottom of the page
dalex
Posted 2019-04-28 5:11 AM (#20805 - in reply to #20803)
Subject: Re: Unable to access site...now problems



Veteran

Posts: 271
100
Location: behind the 4th wall
O no, what a horrible situation! I hope they are able to get things fixed properly. I cannot imagine how much work it will be for the admins and ubers if the whole site stays permanently reset to two months ago.
Top of the page Bottom of the page
dustydigger
Posted 2019-04-28 11:51 AM (#20809 - in reply to #20805)
Subject: Re: Unable to access site...now problems



Elite Veteran

Posts: 1004
1000
Location: UK
Oh dear,I wished I had read these posts first before doing a major update - from February - of my Pick N Mix .Nearly had a heart attack 10 mins ago when I checked out my challenges,and had supposedly read only 10 books,instead of around 26 or more. I had just thought I hadn't updated for a while. Since I post on several sites,and often just copy and paste to save work,I just assumed I hadn't updated this particular site ..Oh well,eventually - hopefully - it will all be duplicated,no big deal. But thank goodness I have other sites to act as backups. Nightmare if it was the only site.
Top of the page Bottom of the page
gallyangel
Posted 2019-04-30 2:24 AM (#20812 - in reply to #20809)
Subject: Re: Unable to access site...now problems



Uber User

Posts: 833
500
Location: The Wilds of Washington
So, any idea when the recovery process might be complete, so we can then move on to reentering everything that has been lost? Or are we still in a holding pattern?
Top of the page Bottom of the page
Administrator
Posted 2019-04-30 2:43 AM (#20813 - in reply to #20812)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas
gallyangel - 2019-04-30 2:24 AM So, any idea when the recovery process might be complete, so we can then move on to reentering everything that has been lost? Or are we still in a holding pattern?

Still holding at the moment. We put up a temporary info page late last night here:  http://www.worldswithoutend.com/notice.asp  and added a link in the sidebar.

We have a backup from March 31st that will be going up soon and we'll let everyone know here in this thread when that is complete. Fingers crossed.

Top of the page Bottom of the page
rretzler
Posted 2019-05-01 11:33 AM (#20815 - in reply to #20813)
Subject: Re: Unable to access site...now problems



Member

Posts: 30
25
Location: Dublin, OH
I've been through a virus taking down a server at a small company and the backup not being reliable, even though it was tested weekly. It is a horrible situation and I hate that anyone has to go through anything similar. We eventually got through it, restoring what we could and recreating what we had to. It's a process. Hang in there - you guys are doing a great job!
Top of the page Bottom of the page
Administrator
Posted 2019-05-01 9:22 PM (#20816 - in reply to #20815)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas

rretzler - 2019-05-01 11:33 AM I've been through a virus taking down a server at a small company and the backup not being reliable, even though it was tested weekly. It is a horrible situation and I hate that anyone has to go through anything similar. We eventually got through it, restoring what we could and recreating what we had to. It's a process. Hang in there - you guys are doing a great job!

Thanks for the kind words. It’s a horrible mess for sure and the site keeps going down which adds more uncertainty to the process. We’ve got to get away from this dumpster fire so we can rebuild. 

Top of the page Bottom of the page
cynesthesia
Posted 2019-05-04 1:51 PM (#20817 - in reply to #20786)
Subject: Re: Unable to access site...now problems



New User

Posts: 1
0
Simply, good wishes to you all on dealing with the aftermath. I'm not always aware how much I value this site until there's a problem, so thanks.
Top of the page Bottom of the page
Adele1967
Posted 2019-05-08 7:49 AM (#20818 - in reply to #20817)
Subject: Re: Unable to access site...now problems



Member

Posts: 36
25
Location: Windsor, Ontario
Is it safe to re-enter all our BookTrackr info? or is the still a possibility of a more recent back-up?
Top of the page Bottom of the page
Administrator
Posted 2019-05-08 8:42 AM (#20819 - in reply to #20818)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas
Adele1967 - 2019-05-08 7:49 AMIs it safe to re-enter all our BookTrackr info? or is the still a possibility of a more recent back-up?
We are still struggling to get the site moved to a new host. A2 has gone to drek completely and we can?t get the support we need. I imagine most of their clients are also trying to move away which would is why we get automated responses instead of actual support help. We have found a new host and are trying to move but we have no way to estimate when we can accomplish that. We?re still in standby mode and I would caution everyone to wait a bIt longer. Lame, I know.
Top of the page Bottom of the page
gallyangel
Posted 2019-05-21 11:16 PM (#20827 - in reply to #20819)
Subject: Re: Unable to access site...now problems



Uber User

Posts: 833
500
Location: The Wilds of Washington
Since it's been almost 2 weeks since your last post, I wondered if you had an update for us. I know my to-be-added list is growing.


Edited by gallyangel 2019-05-21 11:17 PM
Top of the page Bottom of the page
Thomcat
Posted 2019-05-22 12:59 PM (#20828 - in reply to #20827)
Subject: Re: Unable to access site...now problems



Member

Posts: 44
25
Location: Seattle, WA
I was about to post the same question, but adding it's been a month since the original failure.

Hoping things are going well at WWE HQ!
Top of the page Bottom of the page
Administrator
Posted 2019-05-22 3:55 PM (#20829 - in reply to #20828)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas

Real life bulldrek keeps getting in the way but we're getting closer. We're ready to transfer the site to our new host service then we'll load the data from the last backup we have. Can't offer a date or anything specific but it should be soon-ish. Crossed fingers and patience are still required.

We have been fixing a bunch of stuff around the site in the mean time so not all wasted time though it feels like it a bit.

Top of the page Bottom of the page
Dlw28
Posted 2019-05-23 7:36 PM (#20830 - in reply to #20819)
Subject: Re: Unable to access site...now problems



Extreme Veteran

Posts: 305
100
Crossing fingers with you!
Top of the page Bottom of the page
mgkurilla
Posted 2019-05-23 7:57 PM (#20831 - in reply to #20829)
Subject: Re: Unable to access site...now problems



Regular

Posts: 74
25
When you say "last back we have" is this the same as the backup you put up that went through March or is there a more recent one such that if we add read books, they will be included in the transfer?

I can understand not wanting to deal with new additions to the database, but what about users' updates to book tracker?

Good luck with the transfer; let's hope you new host is better prepared.
Top of the page Bottom of the page
Rhondak101
Posted 2019-05-24 8:27 AM (#20832 - in reply to #20786)
Subject: Re: Unable to access site...now problems



Uber User

Posts: 770
500
Location: SC, USA
Mgkurilla, In my understanding the reboot will take WWEnd back to March, so any book additions, book tracking, member book reviews, or changes to challenges occurring after that March backup was created will be lost.
Top of the page Bottom of the page
Administrator
Posted 2019-06-03 4:58 PM (#20835 - in reply to #20832)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas

Hello again everyone. The site has been down again from a database issue with our craptastic host service. This time almost exactly a week. Sheesh.

The good news is that we have our new host and site setup in hand and we've just been waiting on the A2 AHoles to fix our data so we can transfer it to the new host. We'll kick that off as soon as Chris gets home from work so he can monitor and test. Once that is done we'll flip the switch to the new host. Which will take the site down yet again but only for a few hours.

It is our hope that this will be the last of the outages and we can begin the long road to recovery for all that lost data. Going to be a busy week for me getting that sorted. More news as we have it. Thanks as always for your patience and all the good wishes we got from friends of WWEnd via email and text etc. Ya'll rock!

Top of the page Bottom of the page
rretzler
Posted 2019-06-03 7:46 PM (#20837 - in reply to #20819)
Subject: Re: Unable to access site...now problems



Member

Posts: 30
25
Location: Dublin, OH
Crossing my fingers that it all goes well!
Top of the page Bottom of the page
Administrator
Posted 2019-06-06 10:26 AM (#20839 - in reply to #20837)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas

Finally have some good solid news. The site is set up on the new host in a Virtual Private Network that should speed things up a bit. Friday night we will shut down the forum and the site will go down for a few hours while we make the final transfer of the data over to the new host.

By Saturday morning we should be up and running again on the new host and we can put this sorry period behind us and begin the real work of repairing the lost data. Wish us luck!

Top of the page Bottom of the page
kristibpatterson
Posted 2019-06-06 10:33 AM (#20840 - in reply to #20839)
Subject: Re: Unable to access site...now problems



Member

Posts: 18
0
Location: Atlanta, GA
Thanks for all your hard work. Best of luck on a smooth end to this adventure.
Top of the page Bottom of the page
Weesam
Posted 2019-06-06 7:16 PM (#20841 - in reply to #20839)
Subject: Re: Unable to access site...now problems



Uber User

Posts: 593
500
Location: New Zealand
Yah! Good luck.
Top of the page Bottom of the page
justifiedsinner
Posted 2019-06-08 9:09 AM (#20845 - in reply to #20841)
Subject: Re: Unable to access site...now problems



Uber User

Posts: 794
500
So, are we back or what?
Top of the page Bottom of the page
Administrator
Posted 2019-06-08 10:07 AM (#20846 - in reply to #20845)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas
Not yet. We discovered an upgrade opportunity that will make the site run a little faster so we are waiting on a software install. It may be this morning but we may have to wait until Monday. It’s better to do it now than take the site down again next week. Should be worth the wait. We appreciate your continued patience. More info here when I get it. 
Top of the page Bottom of the page
Administrator
Posted 2019-06-08 9:15 PM (#20848 - in reply to #20786)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas
Good news! We got the upgrade finished today so we?re lined up to make the switch tomorrow afternoon. Plan is to kick off around 2pm.
Top of the page Bottom of the page
Adele1967
Posted 2019-06-09 8:07 AM (#20849 - in reply to #20786)
Subject: Re: Unable to access site...now problems



Member

Posts: 36
25
Location: Windsor, Ontario
Excellent! Thanks for all the updates.
Top of the page Bottom of the page
Administrator
Posted 2019-06-10 11:13 AM (#20850 - in reply to #20849)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas
We are back up and live again on our new host!

That means we can start inputting new data again without fear of loss. If you've been holding back on tagging books or submitting reviews etc. you can get back to business as usual. That includes new book requests etc.

There is a bit of cleanup to do on the site. It looks like some of the updates I did some weeks ago are not showing on the new site so I'll have to check the CSS and work out what's missing. Unfortunately we had a big storm roll through Dallas on Sunday and I am without power at home. It's like the universe wants to thwart me at every turn. When I get power back on I'll start those fixes.

Please let us know here in this thread if you run into any problems. I guess we might as well start an errors list here too.

Top of the page Bottom of the page
valashain
Posted 2019-06-10 1:42 PM (#20851 - in reply to #20850)
Subject: Re: Unable to access site...now problems



Uber User

Posts: 1465
1000
Location: The Netherlands
Is this all the data we're going to get back Dave? I think there are about a thousand record numbers missing. If that is correct it is going to take me a while get back to where we were.
Top of the page Bottom of the page
Administrator
Posted 2019-06-10 2:22 PM (#20853 - in reply to #20851)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas

valashain - 2019-06-10 1:42 PM Is this all the data we're going to get back Dave? I think there are about a thousand record numbers missing. If that is correct it is going to take me a while get back to where we were.

Yeah, that's all we've got, unfortunately. It's going to take a long time for sure but at least we still have all the book covers and author pics etc. That will help a bit. We should probably open up a discussion in the Uber thread about coordinating efforts etc. I know some other Ubers have started missing content lists already. I'd hate to duplicate any records in our haste to get back to normal.

Top of the page Bottom of the page
valashain
Posted 2019-06-10 3:09 PM (#20854 - in reply to #20853)
Subject: Re: Unable to access site...now problems



Uber User

Posts: 1465
1000
Location: The Netherlands
Hmm, near as I can tell it is some 400 pieces of short fiction and another couple of hundred associated collections and anthologies. I'm considering just finding a convenient exit and just not letting that go. Recreating it would be a godawful lot of work and there doesn't seem to be all that much enthusiasm for it. Or willingness to actually add short fiction in meaningful quantities for that matter.
Top of the page Bottom of the page
Administrator
Posted 2019-06-10 3:33 PM (#20855 - in reply to #20854)
Subject: Re: Unable to access site...now problems



Admin

Posts: 3942
2000
Location: Dallas, Texas

valashain - 2019-06-10 3:09 PM Hmm, near as I can tell it is some 400 pieces of short fiction and another couple of hundred associated collections and anthologies. I'm considering just finding a convenient exit and just not letting that go. Recreating it would be a godawful lot of work and there doesn't seem to be all that much enthusiasm for it. Or willingness to actually add short fiction in meaningful quantities for that matter.

It really is a ton of work and I understand your reluctance to take it on again. I'll help you with that as much as I can and I'm sure we can find some other Ubers who'd be willing to pitch in too. No reason you'd have to do it all by yourself the second time around. Maybe parcel it out into lists of 100 missing shorts to any volunteers? I'll certainly take up the challenge.

I have a lot of enthusiasm for the shorts you've been putting in and I think others will get more enthused when the new site goes up. The treatment for short fiction is much nicer in 3.0 and we have all those short categories to add to the awards to look forward to. I'm looking forward to more short fiction reading challenges and the like as we get deeper in those categories.

The other thing to keep in mind is that we have discouraged folks from requesting short fiction so your project was the only real source of shorts going in. I always looked at your project as setting up a nice foundation of the top short fiction that we would build on more actively once 3.0 launches.

Top of the page Bottom of the page
illegible_scribble
Posted 2019-06-10 3:43 PM (#20857 - in reply to #20786)
Subject: Re: Unable to access site...now problems



Uber User

Posts: 1057
1000

 

I've got a list of many of the missing books, but not most of the short fiction. Give me a bit and I'll get that posted.

 

 

 

Top of the page Bottom of the page
valashain
Posted 2019-06-10 3:44 PM (#20858 - in reply to #20857)
Subject: Re: Unable to access site...now problems



Uber User

Posts: 1465
1000
Location: The Netherlands
The short fiction is easy to trace for me. I have most of it in a spreadsheet.
Top of the page Bottom of the page
Naomi_uk72
Posted 2019-06-11 10:06 AM (#20860 - in reply to #20858)
Subject: Re: Unable to access site...now problems



Veteran

Posts: 116
100
Location: Sheffield, UK
Hi there

Just popping by to say a big thank you and well done to all involved for getting the site through the recent problems. Hopefully with the new hosts this will be the beginning of a grand new era for WWE. :-)

Naomi
Top of the page Bottom of the page