Thank you, everyone, for your patience during our server move. I know it must have been nerve-wracking waiting for your site to return. But now all the files have been copied, all the databases exported and imported, and all the configurations updated.

You should find the site to be faster and more responsive now. Many caching techniques have been put into place, making our server load lighter. Therefore it should be speedier and more reliable from now on.

If you are a forum member, you can read more about our server move by logging on and reading the thread located at [ http://www.digitpress.com/forum/showthread.php?171357 ].

-Nz17 [ Link ]

Working with our security team over at Sucuri, the hack which was affecting the Web site has been purged from our site. The exploit which the hackers were using has been patched so they can no longer use it. All of the hackers' files and backdoors have been deleted so they cannot return.

The hackers had used a flaw in our forum software to gain access to our server. Once there, they left behind lots of files which were a combination of PHP and JavaScript which they then used as a remote console to the server. This allowed them to run commands from afar. The names of the files were randomly chosen, they were given random time stamps to make them seem older and legit, and they were hidden within directories with lots of files, making finding and deleting them all like looking for needles in a haystack.

They had also set up their scripts to make any removal attempts at cleaning our JavaScript files to be futile as whenever their automated scripts updated their injected code, they would just prefix it to all of our .js files yet again.

But now the menace is over and the scourge has been scoured from our site. Please resume your retro gaming activities.

-Nz17 [ Link ]

The various parts of the Digital Press Web site (forums, Room of Doom, etc.) have been locked down in anticipation of a server move. We are changing hosting providers from DreamHost to HostGator. We anticipate that this will alleviate many of the problems which we have been experiencing lately.

In order to have enough time to move all of the domain names and the tens of gigabytes of files, databases, and configurations over to our new host and ensure no data loss, we will need to put the forum and all the rest into standby mode and take a few days to move everything. We believe this will not take any longer than a week. So expect the Digital Press collective to be unavailable starting April 5 until (at the latest) April 11, though we do hope to be back sooner than that by a few days.

Thank you for your patience and understanding.

-Nz17
[ Link ]

We've been experiencing a difficult-to-remove hack on the Digital Press Web sites for the last few days. Not only is it difficult to determine the source of this activity, but the attacker's method is sophisticated enough to also change file permissions and hide the time stamps of activity. This attack affects both publicly viewable files on the Web as well as files which would normally be out of range of public view.

SSH passcodes and public keys have been changed to no avail. We have deleted software and data files which were discovered to have been affected and replaced them with known good backups or new copies, but nonetheless the attack doesn't relent. Thankfully forum passwords are not affected nor viewable as they are hashed to obscure them.

Though we have tried many approaches thus far, we haven't yet exhausted our options. We will continue to try additional measures until we have eliminated this scourge. As a result, visitors might experience intermittent availability of files or services as we examine various parts of the system.

We apologize for this inconvenience and intend to get everything back to normal as soon as possible. Thank you for your patience. -Nz17
[ Link ]

The Room of Doom is still causing memory allocation and database locking problems. Unfortunately our host, DreamHost, isn't being much help in the way of fixing this. The company appears to want to take a reactive rather than preventative approach.

One solution we at DP have considered is to upgrade our Room of Doom software from Gallery2 to Gallery3. But it isn't a simple, drop-in procedure, and something with our database is preventing the automatic importation into Gallery3 from happening. So we are trying a different approach: we are upgrading our image processing software suite.

ImageMagick, which we are currently using for the Room of Doom, is great, but the version of it which DreamHost provides is woefully out-of-date. As a matter of fact, it is four years out of date, and we believe it is probably buggy to boot... and that these bugs are the cause of the image processing requests never finishing on photo uploads and thus the database threads being locked indefinitely, eventually exhausting our supply of connections.

Therefore we have replaced the host-provided copy of ImageMagick with a fresh, just-released-this-month copy from the ImageMagick developers themselves. I have configured, compiled, and installed the software suite and the Room of Doom is now configured to use it. Hopefully this will be the end of our "too much memory used" and locked database errors. -Nz17
[ Link ]

Our host (DreamHost) limits our server to a total of 300 MiB of RAM so we need to watch our memory usage. However our host also foolishly allows processes to grow up to 90 MiB per thread, meaning that just 4 simultaneous threads can exhaust our allocated supply. This will often cause long delays or error messages when trying to visit the site (as the server is waiting for the threads to finish their work).

To deal with this, I have manually changed the configuration of the 'Room of Doom' photo gallery and the 'Retrogaming Roundtable' forum to cause threads to be allocated only 24 MiB each. This will allow for many more threads to simultaneously run and even potentially use up their entire allocation of memory without causing stability problems. -Nz17
[ Link ]

A table crashed on the 'Room of Doom' gallery database, making it unavailable for the past three days. Though nobody informed me of this, I discovered it today and repaired it. The 'Room of Doom' section is once again accessible. Please feel free to post your photos. -Nz17
[ Link ]

Further investigations have revealed that the MySQL databases were never moved away from the MySQL VPS. Our host had denied the move twice due to the large size of the forum database (6.2 GiB) but never notified us of this. Therefore we were operating under the incorrect surmise that we had been moved, then moved back.

Furthermore the true cause of our database troubles lied with the Room of Doom. A few database requests to it were taking too long to complete but wouldn't quit, causing them to live several days each instead of only a few seconds. Over time these added up to too many active connections all at once, making it impossible to allow any new connections to be made. All of the old "zombie" connections have now been killed and database behaviors have returned to normal. Therefore all Web site sections should now be operating correctly once again. -Nz17
[ Link ]

Starting earlier in the week, some odd forum issues were noticed. "Last post" dates on threads were not matching the "last post" dates on sub-forums. The issue seemed to correct itself after a short amount of time and so it was thought to be a minor hiccup. But then on January 11th things went sour with none of our database-powered sections being able to connect to their MySQL databases. These included sections like the forum, game rarity guide, the technical knowledge base, etc. The error reported when trying to connect to any database was that there were too many simultaneous connections.

Further investigations showed that somehow our databases, which we had recently migrated away from a Virtual Private Server setup on December 13, 2013 (and had been running perfectly fine for the past month), had been moved back to a MySQL VPS at our Web host. Corrections were made to the situation and after an automatic move-and-merge which will occur on Monday (Jan. 13) at midnight, things should be working as expected. We have already contacted our Web host about this and shall continue to investigate and keep an eye on the situation in case of regressions. -Nz17
[ Link ]