Tunehouse

User Tag List

Page 3 of 3 FirstFirst 123
Results 41 to 50 of 50

Thread: Unplanned outage

  1. #41
    Nexus's Avatar
    Nexus is offline Administrator
    Join Date
    Nov 2008
    Location
    Geelong
    Age
    45
    Posts
    1,492
    Blog Entries
    1

    Default

    Discovered this afternoon that one of the two gateways that your browser reaches the website through was playing up (VIC gateway), and doing so in such a way that your browsers were not falling back to the gateway that was working (QLD)
    As a result up to 50% of users trying to browse the website may have seen timeout or connection reset messages. Not sure how long this problem had been occurring - I suspect a day or two - as I hadn't seen the symptoms myself. Issue identified and reported through Facebook group. This is first time we've seen this issue that I can recall. Normally if one gateway is down, you should still be able to browse the website. In this case the gateway wasn't down in a way that caused browsers to fail over to the alternative gateway.Thanks Adrian for reporting the problem.

    VIC gateway has been restarted and is behaving normally again.
    "Blue Meanie" 2007 Aurora Blue MPS 3 - 18x8.5+44 SSR GTX01 - 235/40R18 Federal RS-RR - 3.5" ETS TMIC - CPE stg 2 mount - HKS/CPE BPV - 2XS inlet - 2XS short shift - 2XS turbo manifold - Hypertech tune - Leather/Aluminium handbrake - Momo shifty knob - 7" touchscreen - JDM Mazda Retractable dashtop screen assembly - PC based GPS and instrumentation - 36AH reserve battery and C-TEK isolator - TEIN Street Advanced coilovers 1" drop - Superpro bushings - 220Kw/410Nm.

    "Lipstick" 2013 Velocity Red MPS 3 - 18x7.5+48 Enkei RPF1 -225/40R18 Federal RS-RR - CPE TMIC - COBB inlet - CPE stg 2 mount - COBB Stage 1 98 octane tune - COBB shifty knob - 2XS short shift.

  2. #42
    Nexus's Avatar
    Nexus is offline Administrator
    Join Date
    Nov 2008
    Location
    Geelong
    Age
    45
    Posts
    1,492
    Blog Entries
    1

    Default

    internal network link failure between QLD and VIC caused similar faults to the gateway issue identified yesterday. Back online. Outage for 50% of users was from approx 9:20AM to 10:15AM AEDT today

  3. #43
    Nexus's Avatar
    Nexus is offline Administrator
    Join Date
    Nov 2008
    Location
    Geelong
    Age
    45
    Posts
    1,492
    Blog Entries
    1

    Default

    Repeat gateway issue this afternoon. Am suspecting that this may be due to an attack on the gateway, OR some of the small changes to the website in preparation of forum update. If it's related to website adjustments, it's only affecting one gateway (which are not actually part of the website, they just direct your incoming browser requests to the server from one of two public IP addresses), so this seems unlikely as the gateways are near identical.

    Outage affecting 50% of users from about 1:15PM to 2:35PM AEST.

    Will keep an eye on, I will probably attempt the forum update again tonight. After the update the forum config can be returned to prior to update. There are options that need to be turned on for the update to process successfully.
    "Blue Meanie" 2007 Aurora Blue MPS 3 - 18x8.5+44 SSR GTX01 - 235/40R18 Federal RS-RR - 3.5" ETS TMIC - CPE stg 2 mount - HKS/CPE BPV - 2XS inlet - 2XS short shift - 2XS turbo manifold - Hypertech tune - Leather/Aluminium handbrake - Momo shifty knob - 7" touchscreen - JDM Mazda Retractable dashtop screen assembly - PC based GPS and instrumentation - 36AH reserve battery and C-TEK isolator - TEIN Street Advanced coilovers 1" drop - Superpro bushings - 220Kw/410Nm.

    "Lipstick" 2013 Velocity Red MPS 3 - 18x7.5+48 Enkei RPF1 -225/40R18 Federal RS-RR - CPE TMIC - COBB inlet - CPE stg 2 mount - COBB Stage 1 98 octane tune - COBB shifty knob - 2XS short shift.

  4. #44
    Nexus's Avatar
    Nexus is offline Administrator
    Join Date
    Nov 2008
    Location
    Geelong
    Age
    45
    Posts
    1,492
    Blog Entries
    1

    Default

    Having some wet weather related issues with the copper internet line in VIC. Short lived outages - if you have difficulty accessing the site, please retry in a few minutes time.
    If you find the site appears down for longer than 15 minutes, please report issue to ozmpsclub facebook page

  5. #45
    Nexus's Avatar
    Nexus is offline Administrator
    Join Date
    Nov 2008
    Location
    Geelong
    Age
    45
    Posts
    1,492
    Blog Entries
    1

    Default

    Server restarted due to software fault this morning ~11:20AM. Cause unclear but suspected to be an attack on the web service.

  6. #46
    Nexus's Avatar
    Nexus is offline Administrator
    Join Date
    Nov 2008
    Location
    Geelong
    Age
    45
    Posts
    1,492
    Blog Entries
    1

    Default

    Have had a day or two of slow website performance thanks to an update to the reverse proxy gateways inadvertently breaking it's internet route, effectively taking it offline - we've been operating on the QLD gateway only while this fault was present.

  7. #47
    Nexus's Avatar
    Nexus is offline Administrator
    Join Date
    Nov 2008
    Location
    Geelong
    Age
    45
    Posts
    1,492
    Blog Entries
    1

    Default

    Website down due to full storage from sometime after 1:00AM 11/08/17 through to 12:30PM 11/08/17 AEST
    Logs had blown out and filled the storage. Resolved by archiving the logs and deleting from the webserver host.

  8. #48
    Nexus's Avatar
    Nexus is offline Administrator
    Join Date
    Nov 2008
    Location
    Geelong
    Age
    45
    Posts
    1,492
    Blog Entries
    1

    Default Registration fault

    Hi everyone, registration of new users appears to be failing at present - it's unclear why or if it effects all registration tools or just manual registration (without using social media account) I'll be investigating tonight. There was what appeared to be an attack on the site a week or so ago, or it may be more benign such as a bug with the social media registration tools. Will update when I have discovered more about the problem.

  9. #49
    Nexus's Avatar
    Nexus is offline Administrator
    Join Date
    Nov 2008
    Location
    Geelong
    Age
    45
    Posts
    1,492
    Blog Entries
    1

    Default Registration page restored

    The registration page was failing due to faulty human verification. Recapcha was enabled, which is an external service that asks you to select all images with roads in them and various similar checks to verify you are a human and not a bot. As this was failing, I have disabled it and replaced the feature with a simpler text-image check i.e. type what you see in this image - to verify you are human.

    Registration is expected to be fully functional again.

  10. #50
    Nexus's Avatar
    Nexus is offline Administrator
    Join Date
    Nov 2008
    Location
    Geelong
    Age
    45
    Posts
    1,492
    Blog Entries
    1

    Default

    Outage this morning due to power and internet outages at VIC datacenter.

Page 3 of 3 FirstFirst 123

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •