[Kelvin-users] [Lonsdale-users] Cluster Downtimes August 2017

[Sean McGrath]

Dear all,

Again, my apologies if you receive this mail multiple times.

The clusters where restored to production yesterday after the power
interruption and are available again.

Please remember that the next power outage affecting the Lloyd is next Monday
and the following HPC Clusters:

- Lonsdale
- Kelvin
- Parsons
- TCIN

Will be unavailable from 8am Monday 4th until 11am Tuesday 5th of September.

Unfortunately this power interruption lead to a recurrence of the issue from May
that affected the power distribution units in Lonsdale. This means that a large
proportion of the Lonsdale nodes cannot be powered on and Lonsdale is running at
significantly reduced capacity. The additional mitigation steps we took in an
attempt to prevent this problem repeating where unsuccessful.

Our apologies for the inconvenience and gratitude for your patience.

Regards

Sean McGrath



On Thu, Aug 24, 2017 at 03:47:47PM +0100, Sean McGrath wrote:

> Dear all,
> 
> Apologies if you receive this mail multiple times.
> 
> We have been notified of updated times for the power outages affecting the Lloyd
> Building and the cluster down times will be as follows:
> 
> 1) 8am Monday 28th until 9am Wednesday 30th of August
> 
> And
> 
> 2) 8am Monday 4th until 9am Wednesday 6th of September 
> 
> The provisional date of August 24th, (today), was incorrect in the end.
> 
> Our apologies for the inconvenience and gratitude for your patience.
> 
> Regards
> 
> Sean
> 
> On Mon, Aug 21, 2017 at 10:11:18AM +0100, Sean McGrath wrote:
> 
> > 
> > Dear all,
> > 
> > Again, apologies if you receive this mail multiple times.
> > 
> > The power outage that was due to happen today has been postponed. 
> > 
> > We have been advised of a provisional rescheduled date of 8pm Thursday
> > 24th August until 06:45am Friday 25th August 2017. But those dates have
> > not been confirmed.
> > 
> > Our apologies for the inconvenience this will have. These changes are
> > outside of our control and we will try to keep you updated as best we
> > can.
> > 
> > Regards
> > 
> > Sean
> > 
> > On Wed, 2017-08-16 at 17:36 +0100, Sean McGrath wrote:
> > > Dear all,
> > > 
> > > Apologies if you receive this mail multiple times.
> > > 
> > > As part of the Universities Medium Voltage Infrastructure Upgrade
> > > Project there will be a loss of power to the Lloyd Building which will
> > > affect the Data Centre on the following dates:
> > > 
> > > 1) 8pm Monday 21st August 2017 until 06:45am Tuesday 22nd August.
> > > 
> > > 2) 2pm to 2:01pm Tuesday 31st August 2017.
> > > 
> > > This loss of power will necessitate the powering off of all affected
> > > equipment in the Lloyd Data Centre in advance of those times. It will
> > > take us some time to return all services to production once power is
> > > restored. We apologise for the inconvenience that this necessary work
> > > will cause. 
> > > 
> > > The following HPC Clusters will be affected:
> > > 
> > > - Lonsdale
> > > - Kelvin
> > > - Parsons
> > > - TCIN 
> > > 
> > > Reservations have been placed in the queue's on each of the clusters as
> > > follows:
> > > 
> > > - 9am on the 21st of August until 9am on the 23rd of August.
> > > - 9am on the 31st of August until 9am on the 2nd of September.
> > > 
> > > The length of the downtime is required as there are a large number of 
> > > systems in the Data Centre that also need to powered off and back on by
> > > TCHPC. We would hope to return the clusters to service as soon as
> > > possible once power is restored.
> > > 
> > > Further updates on this will be posted to
> > > https://www.tchpc.tcd.ie/node/1287
> > > 
> > > We apologise for the inconvenience that this necessary work will cause.
> > > 
> > > Regards
> > > 
> > > Sean McGrath
> > > 
> > > _______________________________________________
> > > Parsons-users mailing list
> > > Parsons-users at lists.tchpc.tcd.ie
> > > https://lists.tchpc.tcd.ie/listinfo/parsons-users
> > > 
> > 
> 
> -- 
> Sean McGrath M.Sc
> 
> Systems Administrator
> Trinity Centre for High Performance and Research Computing
> Trinity College Dublin
> 
> sean.mcgrath at tchpc.tcd.ie
> 
> https://www.tcd.ie/
> https://www.tchpc.tcd.ie/
> 
> +353 (0) 1 896 3725
> 
> _______________________________________________
> Lonsdale-users mailing list
> Lonsdale-users at lists.tchpc.tcd.ie
> https://lists.tchpc.tcd.ie/listinfo/lonsdale-users
> 

-- 
Sean McGrath M.Sc

Systems Administrator
Trinity Centre for High Performance and Research Computing
Trinity College Dublin

sean.mcgrath at tchpc.tcd.ie

https://www.tcd.ie/
https://www.tchpc.tcd.ie/

+353 (0) 1 896 3725