Jump to content


Network issue on our dedicated servers


  • Please log in to reply
6 replies to this topic

#1 Guest_raja.k_*

Guest_raja.k_*
  • Guests

Posted 17 April 2014 - 05:11 AM

This is a continuation of http://forums.myorde...icated-servers/

 

-------------------------------------------------------------------------------------------------------------------------------------------------------

 

As of now, the connectivity issues for our Dedicated Servers still persist. As per the last update from the data center, intense investigations are being carried out to fix this issue. However, we are unable to give an ETA at this point as we have been unable to determine the root cause of this issue so far.

Appropriate alternate solutions are being contemplated to fix the issue as soon as possible. These solutions include replacing the core routers if need be.

This issue is being worked upon on the highest priority and we will update the forums on a regular basis with the details on the progress. You can contact our support help desk via phone / ticket in case you have further doubts / issues.

We regret the inconvenience caused due to this.

 

-------------------------------------------------------------------------------------------------------------------------------------------------------



#2 Guest_alisha.d_*

Guest_alisha.d_*
  • Guests

Posted 17 April 2014 - 06:07 AM

Our Network Operations team is still working to resolve the issue with the routers and to restore services for all affected customers. Some sites are coming back online, but as of now they cannot yet provide an ETA. As we get more information, we'll continue to provide updates here on the forums till everyone affected has their services restored.

We assure you that work on the current issue will continue until a full resolution is achieved. This remains our absolute (and only) priority until all services are fully restored.

 

We appreciate your patience as we work through this issue.



#3 Guest_alisha.d_*

Guest_alisha.d_*
  • Guests

Posted 17 April 2014 - 07:27 AM

Services have been restored to many customers, however some may still experience service instability. We appreciate your patience while we work towards a full resolution.



#4 Guest_alisha.d_*

Guest_alisha.d_*
  • Guests

Posted 17 April 2014 - 08:42 AM

We have successfully isolated the root cause of this issue. This was caused due to a router hardware component failure at the datacenter. A fix has been identified and is currently being deployed across the network. You might encounter a few minor issues till this fix is fully implemented. However, restoring services and functionality from here on should be a speedy process.

Most of our servers are already showing as completely up and running. Once we confirm the status of all servers, we'll post a detailed analysis of the downtime.

We sincerely apologize for the inconvenience that you and your Customers faced due to this. Rest assured, we'll do everything possible from our side to ensure that this does not happen again.



#5 Guest_vicky.k_*

Guest_vicky.k_*
  • Guests

Posted 17 April 2014 - 04:45 PM

Although all primary IPs of the servers are up, a few addon IPs are still facing some lingering issues.

As mentioned earlier, this issue was due to a bug in our vendor's hardware. This has been addressed now. The fix has been applied to all primary IPs and most add on IPs. If you are facing any issues with your sites or hosted applications then please do get in touch with our support team now.

We understand that this has caused a lot of inconvenience to you and your clients. However, we assure you that our team is here to help you out and will address all tickets raised with the highest of priority.



#6 Guest_Kayalvizhi Subramaniam_*

Guest_Kayalvizhi Subramaniam_*
  • Guests

Posted 18 April 2014 - 06:08 AM

All the additional IP addresses are now up and responding fine to network requests. If you have blocked ICMP on the server, please try to telnet to port 80 for the IP address using the command "telnet ipaddress 80" where "ipaddress" should be replaced by the actual IP address.
 
If you are still facing issues, please feel free to contact our support team. Thank you for your co-operation and understanding.


#7 Guest_alisha.d_*

Guest_alisha.d_*
  • Guests

Posted 23 April 2014 - 02:38 PM

The recent disruption encountered on our Dedicated Servers services was due to a network outage that has been fixed.

 

We undertook a thorough check of all systems and processes at our end to ensure that such an issue does not reoccur. We understand that you may have several questions regarding this issue, here is a breakdown of what happened.

 

- Network services at our data center was affected due to an undocumented firmware bug in one of our vendor's hardware solutions

- It took longer than usual to diagnose the issue as there was no precedent of the bug and we had to scan our downstream systems and entire network to ensure that this issue does not reoccur

- We deployed a firmware update to the systems as a remedy to the problem

- Full services were restored immediately post the firmware upgrade, for the customers

- All the primary IPs were up, and the issues that were later on faced on add-on IPs has been resolved as well

 

We apologize for any downtime that you experienced and the inconvenience the outage caused. The servers continued to operate during this entire period, which means, that at no point in time was your data at risk. The problem was access to the servers because of the firmware issue.

 

We would like to say that we will never have a network service outage again, but realistically that isn't something we can promise. What we can assure you is that we are continually taking steps to audit and improve the performance of our infrastructure, and investing a large amount of capital and people to do this.

 

Thank you for your patience and understanding.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users