Online Auction Solutions > System Status > Data Center Infrastructure Status History

Tuesday, June 15th, 2021 from 10:35 CDT - 14:30 CDT | Cloud Server Incident
  • An incident (CSHD-7e619df5) at the RacksSpace Chicago datacenter disrupted service of our "socket" server. This caused site search to stop working. It also disrupted the live timers and bid updates for lot detail and lot list pages. This meant pages needed be manually refreshed to see new bids and changes in status/close timers.

    As of 14:30 CDT service was restored to normal. Tests indicate that functionality is fully restored. The incident was related to the hypervisor our instance was hosted on. There was no suggestion that this issue might reoccur. In the coming months we will be migrating away from this hosting provider. We will have more redundancy in place after that and should be better able to avoid single points of failure like we experienced with this.

Tuesday, August 4, 2020 from 12:00 AM CDT - 2:00 AM CDT | Network Upgrades
  • We will be undergoing this extended period of downtime to perform regular maintenance to our network infrastructure. During all, or parts of this window, OAS sites will be displaying a 'down for maintenance' page. A half hour prior to the downtime a message will be placed on OAS sites informing users of the pending downtime.

Thursday, July 2, 2020 from 12:30 AM CDT - 3:30 AM CDT | Database Upgrades
  • We will be undergoing this extended period of downtime to perform regular maintenance to our database infrastructure. During all, or parts of this window, OAS sites will be displaying a 'down of maintenance' page.

Friday, February 28, 2020 at 23:00 CST | Hardware Changes
  • Maintenance Window: FRI 23:00 - SAT 07:00 CST
    Hardware migration / DC consolidation. This will result in an extended period of downtime inside the maintenance window.
    These upgrades are necessary to ensure the modernity and stability of our infrastructure.

NOV 18, 2019 | Scheduled SAN Upgrades Pt. 2
  • Maintenance Window: 01:01 - 03:00 CST
    We will be performing upgrades to our database storage array. This will result in an extended period of downtime inside the maintenance window.
    These upgrades are necessary to ensure the modernity and stability of our infrastructure.

NOV 13, 2019 | Scheduled SAN Upgrades
  • Maintenance Window: 12:01 - 03:00 CST
    We will be performing upgrades to our database storage array. This will result in an extended period of downtime inside the maintenance window.
    These upgrades are necessary to ensure the modernity and stability of our infrastructure.

Scheduled Maintenance: 2019-10-10 01:01 CDT - 2019-08-09 03:00 CDT
  • Expected Disruption: Up to one hour of packet loss or momentary losses of connectivity during the maintenance window.

Scheduled Maintenance: 2019-08-09 01:00 CDT - 2019-08-09 01:15 CDT
  • We will be performing some routine updates to our firewall. We expect around five minutes of downtime.

Scheduled Maintenance: 2019-07-12 00:45 CDT - 2019-07-12 01:15 CDT
  • We will be performing a disruptive maintenance on our network infrastructure between 00:45 and 01:15 on July 12, 2019 CDT. We anticipate downtime lasting approximately 15 minutes.

    We apologize for any inconvenience. If you have any questions, please contact us.

April 12, 2018 - 09:00 AM - 09:30 PM (CST) - Network Incident
  • Update - Apr 12'th 2018 - 14:30 CST We have fixed the network issue and site performance has returned to normal. We are still working with our hosting partners on a postmortem of the incident. We do not anticipate further disruption but will likely need to perform some site maintenance overnight (approx 1:00 AM CST).

    Incident - Apr 12'th 2018 - 13:01 CST We are experiencing network issues that have reduced our frontend server availability leading to sluggish performance on OAS sites. We are investigating the issue and will update when we know more.

Tuesday January 30, 2018 - 09:00 AM - 09:30 PM (CST) - Scheduled Downtime / Maintenance
  • We will be performing emergency maintenance on a network gateway which will result in around 3 minutes of downtime sometime during the maintenance window.

Saturday December 23, 2017 - 10:00 AM - 01:30 PM (CST) - Scheduled Downtime / Maintenance
  • We will be performing database server maintenance. During this window we expect a couple of small interruptions (5-10 second blips) to OAS websites. We will not be taking any additional steps to alert users as since the impact will be so minimal.

May 1st, 2017 | 9:00 PM - 10:00 PM CST | OAS Image Tools Unavailable

Image Tools on OAS will be unavailable between approximately 2017-05-01 21:00 CDT and 2017-05-01 22:00 CDT while maintenance is performed on the network. This means you will be unable to upload images to an auction, edit any images, or save/publish any images. Please plan accordingly.

September 8th, 2016 | 11:00 PM - 01:00 AM CST | Database Cluster OS Update

All sites will unavailable during this maintenance. It will take at most two hours to complete. The sites will display a 'down for maintenance page' while the updates are being carried out.

July 17, 2016 | 14:45 - 22:30 CST | Error when Saving on Manage Image page

The problem was tied to segfaults in libfreepriv3.so caused by some components that utilize nss libraries. The issue has been temporarily resolved by downgrading nss. Long term there will be a solution provided by Red Hat or our hosting provider will make adjustments to the OpenStack hypervisor the server instance is running on.

May 17, 2016 | 00:00 - 01:00 CST | Scheduled Maintenance

All OAS sites will be impacted. During the hour there will be periods of downtime. This maintenance is to update the controller firmware on servers in our database cluster. This action is in response to downtime experienced on May 10th. The update is needed to prevent a recurrence of said issue, where, "The Server May Stop Responding and Will Display Power-On Self-Test (POST) Error Messages on Reboot When Running Smart Array Controllers and/or Host Bus Adapters".

May 10, 2016 | 14:34 - 14:14 CST | Site Outage

An incident with our database hardware caused a site outage. Steps are being taken to upgrade firmware to ensure that this same incident does not reoccur.