Bigger, better, faster, more

I debated whether to write a humorous intro, but I’ve ultimately decided it’s more important to get succinct information out to everyone, so here’s the TLDR:
Over the next few weeks, we will migrate NearlyFreeSpeech.NET to all-new equipment and greatly upgraded network infrastructure.

• We’re replacing our Intel Xeon servers with brand-new AMD Epyc servers.
• All our existing file storage will be migrated from SATA SSDs to NVMe PCIe 4.0 SSDs.
• Most of our content will be served from New York City rather than Phoenix after the upgrade.
• Various things may be intermittently weird or slow for the next couple of weeks as we shift them around, but we’re working hard to minimize and avoid disruptions to hosted services.

Continue reading Bigger, better, faster, more…

Hey! What happened to 2023Q2?

You may have noticed that production sites with normal updates are being upgraded from 2022Q4 to 2023Q1, and non-production sites are being upgraded from 2023Q1 to 2023Q3. So what happened to 2023Q2?
Continue reading Hey! What happened to 2023Q2?…

Maintenance for Christmas

Christmas Eve and Christmas Day are the lowest-usage days of the year (both in terms of member activity and in terms of visits to member sites), so we are going to roll out some core system upgrades over the next 36 hours. These updates relate mostly to file servers.

Despite having no single point of failure from the hardware perspective, each site’s content is still backed by a single system image (necessary for coherency), so these updates may cause some temporary disruptions to affected sites. We will do our best to minimize that.

We do also plan to upgrade our core database servers. These are fully redundant, so we do not anticipate disruption, but the possibility does exist. We hope this upgrade will resolve an issue that mainly manifests as intermittent errors in our member interface early in the morning (UTC) on Sundays.

Upcoming updates, upgrades, and maintenance

We have accumulated some housekeeping tasks that we’ll be taking care of over the next couple of months. They’re all necessary things to make sure our service keeps running at its best, and though we work hard to prevent these types of things from impacting services, occasionally they do intrude. As a result, we want to let everyone know what we’re up to and what the effects will be.
Continue reading Upcoming updates, upgrades, and maintenance…

Automatic file server upgrades

As most of our members are aware, one of our older file servers, f5, has been causing intermittent problems. The time has come to move the sites still using it to newer, faster, more reliable equipment. The ability to do that manually has been available in our UI for about a week now, and it has not surprisingly been pretty popular. But after that server caused additional downtime this past week, we’re moving to the next phase: moving sites automatically.
Continue reading Automatic file server upgrades…

Post-mortem report of Saturday’s file server failure

On Saturday, March 29 at about 4pm US Eastern time, we rebooted one of our file servers that hosts content for member sites. It experienced a critical hardware failure and did not come back online. It took about 28 hours to get things back into service. We’re going to talk briefly about why that happened, and what we’ll be doing differently in the future.
Continue reading Post-mortem report of Saturday’s file server failure…

IPv6, SSL, scheduled tasks, storage discounts & bulk bandwidth

We have quite a few feature announcements to bring you this holiday season. We’ve added support for several features, some of which have been requested for years like IPv6, SSL, and scheduled tasks (AKA cron jobs). We’ve also introduced new billing options that make our service pricing fairer and more scalable; these options will help a broad variety of our members save money.
Continue reading IPv6, SSL, scheduled tasks, storage discounts & bulk bandwidth…

Security flaw with login corrected

One of our members informed us a couple of days ago that due to a strange combination of actions and circumstances, he hit a flaw in our login system that enabled him to access the membership of another member with a similar name.

Of course we promptly investigated; the problem has been permanently fixed.
Continue reading Security flaw with login corrected…

Scheduled maintenance November 22 and December 15

We are scheduling two maintenance windows in the next month to move some equipment:

Date: November 22nd, 2010
Window: 9am to noon UTC (4am to 7am US Eastern, 1am to 4am US Pacific)
Affecting: MySQL nodes m2, m3, and m21

Date: December 15th, 2010
Window: 8am to 1pm UTC (3am to 8am US Eastern, midnight to 5am US Pacific)
Affecting: File servers f2 and f5

Each server should be offline for about one hour, not the whole window. This will cause some downtime. While the MySQL nodes are offline, those MySQL processes hosted on them will be unreachable. While the file servers are offline, sites hosted by those file servers will show an official maintenance page.
Continue reading Scheduled maintenance November 22 and December 15…

Removing deprecated IP block

Many years ago, we were assigned the IP address block 64.238.220.0/23 by one of our upstream network providers. We officially deprecated the use of that block way back in 2008, and we will be returning it on December 1st, 2010, so it will not work after that point.
Continue reading Removing deprecated IP block…