Infra.ci.jenkins.io on Jenkins Status Page

[Resolved] Maintenance on weekly.ci.jenkins.io and infra.ci.jenkins.io

Fri, 27 Feb 2026 09:30:00 +0000

[Initial Message] Friday February 27 2026, starting at 09:30 UTC, the Jenkins controllers and agents on weekly.ci.jenkins.io and infra.ci.jenkins.io will be upgraded to use JDK25 at runtime instead of JDK21.

This operation will require controller restarts and agent reconnections. A short service disruption is expected during the maintenance window.

See more details at https://github.com/jenkins-infra/helpdesk/issues/4941

[Resolved] [publick8s cluster] Azure publick8s Upgrade to Kubernetes 1.33.5

Mon, 15 Dec 2025 08:30:00 +0000

[Final Message] upgrade went well, no impact on production.

[Initial Message] Monday 15 December 2025 at 08:30PM UTC, we’ll upgrade the AKS(Azure) cluster publick8s to Kubernetes 1.33.5. It should take between 60 and 120 min.

The expected impacts are:

Restart of the following services, with a downtime between 2 and 15 min (depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-in/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) on:

get.jenkins.io (mirror redirector)
issue-redirect.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
plugin-health.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

More details in https://github.com/jenkins-infra/helpdesk/issues/4820#issuecomment-3548729217.

[Resolved] [privatek8s cluster] Azure privatek8s Upgrade to Kubernetes 1.33.5

Fri, 12 Dec 2025 09:00:00 +0000

[Final message] Upgrade completed without any issue. We’ve done it today only as yesterday GitHub was too unstable to perform it without hitting issues when pulling terraform providers or helm repositories.

[Initial message] Friday 12 December 2025 at 09:00 UTC, we’ll upgrade the AKS(Azure) cluster privatek8s to Kubernetes 1.33.5. It should take between 30 and 60 min.

Expect multiple restarts and downtimes for the infra.ci.jenkins.io and release.ci.jenkins.io controllers along with the github-comment-ops bots.

More details in https://github.com/jenkins-infra/helpdesk/issues/4820#issuecomment-3548723638.

[Resolved] Azure outage on the '*.ci.jenkins.io' private controllers

Wed, 29 Oct 2025 15:45:00 +0000

[Final Message] The outage seems has been resolved at 00:05am on the 30 October.

[Initial Message] Azure is facing a global outage (Tracking ID: YKYN-BWZ at https://azure.status.microsoft/en-us/status/history/) which prevents access to the Azure Portal and to the Azure Identity Management API.

It means that our private controllers might be unable to spin up new VM agents: we expect delays in

The publication of our websites (both previews and production)
Jobs running in private controllers
Execution of infrastructure related tasks (Terraform and Kubernetes jobs)

Jobs will be queued and eventually aborted due to timeouts.

[Resolved] Azure cluster publick8s migrated to a new location and upgraded to Kubernetes 1.32

Thu, 25 Sep 2025 11:30:00 +0000

[Final Message] Outage finished on Friday 26 September 2025 at 09:35am UTC and post mortem published: https://github.com/jenkins-infra/helpdesk/issues/4813#issuecomment-3339518665.

[Update] Friday 26 September 2025 at 09:35am UTC: ongoing production outage on the cluster (unexpected side effect of the migration): https://github.com/jenkins-infra/helpdesk/issues/4813.

[Update] Migration was finished on Thursday 25 September 2025 at 03:30PM UTC. See https://github.com/jenkins-infra/helpdesk/issues/4617#issuecomment-3334826339.

[Initial Message] Thursday 25 September 2025, starting at 11:30AM UTC, we’ll perform a migration + upgrade of the AKS(Azure) cluster publick8s to a new location and to Kubernetes 1.32.

The expected impacts are:

Restart of the following services, with a downtime between 2 and 15 min (depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-in/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) on:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
plugin-health.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

More details in https://github.com/jenkins-infra/helpdesk/issues/4617.

[Resolved] infra.ci.jenkins.io down

Mon, 28 Jul 2025 08:00:00 +0000

[Final message] infra.ci is back to business after a migration to another cluster which allowed to find the root cause and fix it.

[Initial message] The infra.ci.jenkins.io (private) Jenkins controller is down with error related to cloud storage. We are currently analyzing the issue and fixing it.

All website previews in pull requests, along with deployment of plugins.jenkins.io, docs.jenkins.io and stats.jenkins.io are down.

See https://github.com/jenkins-infra/helpdesk/issues/4753 for details.

[Resolved] jdk21 upgrade for runtime controllers and agents

Thu, 22 May 2025 13:00:00 +0000

[Final Message] migration went well, no impact on production. few fixes to apply with https://github.com/jenkins-infra/jenkins-infra/pull/4143

[Initial Message] Thursday May 22 2025, starting at ~~17:00~~ 13:00 UTC, all our controllers and there agents will be upgraded to use JDK21 as runtime version instead of JDK17.

This operation expects 1 or 2 restarts per controller https://github.com/jenkins-infra/helpdesk/issues/4121#issuecomment-2890355054.

[Resolved] LDAP offline for reindex

Tue, 06 May 2025 17:50:00 +0000

The LDAP service requires a reindex which needs LDAP server to be offline. Starting the operation at 17h50 UTC, finished at 18h30 UTC with success.

[Resolved] [privatek8s cluster] Migrate to another Azure subscription

Tue, 29 Apr 2025 08:00:00 +0000

Tuesday 29 April 2025, at 08:00am UTC, the cluster privatek8s will be migrated between Azure subscriptions.

Expect a full downtime of infra.ci.jenkins.io and release.ci.jenkins.io from 2 hours up to 8 hours.

More details in https://github.com/jenkins-infra/helpdesk/issues/4250.

[Resolved] [publick8s cluster] Azure publick8s Upgrade to Kubernetes 1.31.6

Thu, 24 Apr 2025 07:30:00 +0000

[Final Message] upgrade went well, no impact on production.

[Initial Message] Thursday 24 April 2025, at 07:30PM UTC, we’ll upgrade the AKS(Azure) cluster publick8s to Kubernetes 1.31.6. It should take between 60 and 120 min.

The expected impacts are:

⚠️ Last Kubernetes upgrade (to 1.30) we saw an outage on update-center (updates.jenkins.io) for a few seconds !

Restart of the following services, with a downtime between 2 and 15 min (depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-in/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) on:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
plugin-health.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

More details in https://github.com/jenkins-infra/helpdesk/issues/4546#issuecomment-2674545933.

[Resolved] [privatek8s cluster] Azure privatek8s Upgrade to Kubernetes 1.31.6

Wed, 23 Apr 2025 12:30:00 +0000

Wednesday 23 April 2025, at 12:30 UTC, we’ll upgrade the AKS(Azure) cluster privatek8s to Kubernetes 1.31.6. It should take between 30 and 60 min.

Expect multiple restarts and downtimes for the infra.ci.jenkins.io and release.ci.jenkins.io controllers along with the github-comment-ops bots.

More details in https://github.com/jenkins-infra/helpdesk/issues/4546#issuecomment-2674551720.

[Resolved] [publick8s cluster] Azure publick8s Upgrade to Kubernetes 1.30.7

Thu, 20 Feb 2025 13:30:00 +0000

[Final Message] upgrade went well but we saw an outage on update center for about 2 to 3 mn we will analyse further the logs to try to understand were it’s coming from

[Initial Message] Thurday 20 February 2025, at 01:30PM UTC, we’ll upgrade the AKS(Azure) cluster publick8s to Kubernetes 1.30.7. It should take between 60 and 120 min.

The expected impacts are:

Restart of the following services, with a downtime between 2 and 15 min (depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-in/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) on:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
plugin-health.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

More details in https://github.com/jenkins-infra/helpdesk/issues/4258#issuecomment-2663553626.

[Resolved] [privatek8s cluster] Azure privatek8s Upgrade to Kubernetes 1.30.7

Thu, 20 Feb 2025 08:00:00 +0000

[Final Message] Upgrade went well. Had to wait for the infra.ci mounting volume : Warning FailedAttachVolume 5m35s attachdetach-controller Multi-Attach error for volume "jenkins-infra-pv" Volume is already exclusively attached to one node and can't be attached to another but seems faster than before.

[Initial Message] Thurday 20 February 2025, at 08:00 UTC, we’ll upgrade the AKS(Azure) cluster privatek8s to Kubernetes 1.30.7. It should take between 30 and 60 min.

Expect multiple restarts and downtimes for the infra.ci.jenkins.io and release.ci.jenkins.io controllers along with the github-comment-ops bots.

More details in https://github.com/jenkins-infra/helpdesk/issues/4258#issuecomment-2668175339.

[Resolved] ldap outage

Fri, 07 Feb 2025 01:40:00 +0000

[Final Message] The issue was identified to be an undocumented missing permission between Kubernetes and the LDAP data disk during Kubernetes Node upgrade. It has been fixed and persisted.

[Initial Message] ldap service is down so no authentification for Jira, Artifactory, or Jenkins controllers. Help desk 4531 is tracking progress on the issue.

[Resolved] AKS cluster upgrade to 1.29.10 (patch)

Wed, 18 Dec 2024 09:45:00 +0000

We’re performing AKS cluster upgrade from 1.29.7 to 1.29.10 the 18 December 2024, starting at 09:45 UTC.

The expected impacts are:

Restart of the following services, with an outage (between 2 to 15 min: depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-In/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

[Resolved] Azure VMs outage

Mon, 09 Sep 2024 15:00:00 +0000

[Final message]

Azure did fix the problem in two steps:

Additional capacity added in US East 2 the 12 September
Software fix the 13 September

[Update on 2024-09-10]

The problem was gone from 2024-09-09 around 11:00pm UTC but it’s back since 2024-09-10 around 02:00pm UTC
Our Azure Service Health Dashboard mentions 2 mixed incidents, one should be fixed the 12!

[initial message] Since Monday 9 September 2024 at 03:00pm UTC, we see an increased rate of failure to spin up Azure VM agents on ci.jenkins.io

Details in https://github.com/jenkins-infra/helpdesk/issues/4287.

[Resolved] Update Azure AKS cluster `publick8s` to 1.29

Fri, 23 Aug 2024 09:00:00 +0000

Friday 23 August at 07:30am UTC, the AKS cluster publick8s will be upgraded to Kubernetes 1.29.7. It should take between 30 and 60 min.

The expected impacts are:

Restart of the following services, with a downtime between 2 and 15 min (depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-in/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) on:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
plugin-health.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

More details in https://github.com/jenkins-infra/helpdesk/issues/4161.

[Resolved] Update Azure AKS cluster `privatek8s` to 1.29

Fri, 23 Aug 2024 07:30:00 +0000

[Final message]

Upgrade finished with success.

[Initial message] Friday 23 August at 07:30am UTC, the AKS cluster privatek8s will be upgraded to Kubernetes 1.29.7. It should take between 30 and 60 min.

Expect multiple restarts and downtimes for the infra.ci.jenkins.io and release.ci.jenkins.io controllers along with the rss2twitter, ircbot and github-comment-ops bots.

More details in https://github.com/jenkins-infra/helpdesk/issues/4161.

[Resolved] ldap.jio and repo.azure.jio workloads migration to arm64 (publick8s cluster)

Fri, 05 Jul 2024 14:30:00 +0000

We’re doing a workload migration to arm64 on the publick8s AKS cluster starting at 14:30 UTC.

The expected impacts are:

Restart of the following services, with an outage (between 2 to 15 min: depends on Azure Storage systems):

ldap.jenkins.io
repo.azure.jenkins.io (ACP)

Temporary disruptions (no Log-In/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

More details on https://github.com/jenkins-infra/helpdesk/issues/3837#issuecomment-2210532769.

[Resolved] Update azure clusters publick8s Kubernetes version to 1.28

Thu, 04 Jul 2024 06:30:00 +0000

[Final Message] The operation finished successfully at 7:15am UTC.

More details on https://github.com/jenkins-infra/helpdesk/issues/4144#issuecomment-2196274746.

[Initial message] We’re doing an publick8s AZ cluster upgrade from 1.27 to 1.28 tomorrow, 4th of July 2024, starting at 6:30 UTC (8h30 paris time, 12h00 IST).

The expected impacts are:

Restart of the following service, with an outage (between 2 to 15 min: depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-In/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) due to public IP metadata changes and falco + Nginx upgrades:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
plugin-health.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

[Resolved] Maintenance on infra.ci.jenkins.io

Thu, 11 Apr 2024 08:00:00 +0000

[Final message] Migration face multiple issues:

needed PV/PVC creation on the disk from snapshot before cloning by the jenkins helmchart
needed some optimization to deal with timout on volume mount https://github.com/jenkinsci/helm-charts/tree/a052ad9f6fc0f95e67802edc7cef9ab9c3572bee/charts/jenkins#long-volume-attachmount-times

[Initial message]

The infra.ci.jenkins.io controller will be down to migrate the volume used to ZRS as per

https://github.com/jenkins-infra/helpdesk/issues/3823#issuecomment-2036372624

[Resolved] Artifact Caching Proxy deployment of 1.0.0

Thu, 28 Mar 2024 07:30:00 +0000

[Final message]

Upgrade successfull

See https://github.com/jenkins-infra/helpdesk/issues/3969#issuecomment-2024586947 for details.

[Initial message]

The Artifact Caching Proxy system used by ci.jenkins.io and infra.ci.jenkins.io agents to cache Maven artifacts will upgraded to version 1.0.0 the 28 March 2024 at 07:30am UTC.

Expect eventual slowness or failures in Maven dependency resolutions in the next hour on ci.jenkins.io and infra.ci.jenkins.io.

See https://github.com/jenkins-infra/helpdesk/issues/3969#issuecomment-2024586947 for details.

[Resolved] Update azure clusters publick8s Kubernetes version to 1.27

Tue, 19 Mar 2024 08:30:00 +0000

[Final Message] The operation finished successfully at 9:35am UTC.

More details on https://github.com/jenkins-infra/helpdesk/issues/3948#issuecomment-1991181964.

[Initial message] We’re doing an publick8s AZ cluster upgrade from 1.26 to 1.27 today 19th of Mars 2024, starting at 8:30 UTC (9h30 paris time).

The expected impacts are:

Restart of the following service, with an outage (between 2 to 15 min: depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-In/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) due to public IP metadata changes and falco + Nginx upgrades:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

[Resolved] LDAP (ldap.jenkins.io) maintenance

Fri, 19 Jan 2024 08:45:00 +0000

Friday January 19 2024, starting at 08:45am UTC, we’ll proceed to a maintenance on the LDAP service (ldap.jenkins.io) hosted in the publick8s Azure Kubernetes cluster.

Authentication on all of the *.jenkins.io and *.jenkins-ci.orf services will be unavailable during the operation. We expect a disruption between 5 and 30 min.

The goal is to update the LDAP image to a new version to use a new deployment system.

More details in https://github.com/jenkins-infra/helpdesk/issues/3837#issuecomment-1898378430.

[Resolved] Update publick8s AKS cluster to Kubernetes 1.26

Thu, 09 Nov 2023 08:30:00 +0000

[Final Message] The operation finished successfully at 11:00am UTC. The monitoring showed an outage of 6 minutes on the LDAP and 2 minutes on the incremental publisher.

More details on https://github.com/jenkins-infra/helpdesk/issues/3683#issuecomment-1803566680.

[Initial Message] The AKS cluster publick8s will be upgraded from Kubernetes 1.25 to 1.26 the Thursday 09th of November 2023, starting at 08:30am UTC.

The expected impacts are:

Restart of the following service, with an outage (between 2 to 15 min: depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-In/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) due to public IP metadata changes and falco + Nginx upgrades:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

[Resolved] Update privatek8s AKS cluster to Kubernetes 1.26

Tue, 07 Nov 2023 08:00:00 +0000

[Final Message] The operation finished with success at 09:15am UTC

[Initial Message] We’re doing an AKS cluster upgrade from 1.25 to 1.26 the Tuesday 07th of November 2023, starting at 08:00am UTC.

The expected impacts are:

infra.ci.jenkins.io will be down during 20 to 30 min (time for the upgrade to be performed and persistent volumes to be migrated)
No website preview of *.jenkins.io on pull requests
release.ci.jenkins.io will be down during 20 to 30 min (time for the upgrade to be performed and persistent volumes to be migrated)

[Resolved] Brownout of the Maven Central mirrored repositories in JFrog Artifactory (repo.jenkins-ci.org)

Mon, 04 Sep 2023 12:00:00 +0000

[Final Message] Brownout is finished.

[Initial Message]

Monday 04 of September 2023, we’ll perform a brownout operation of 4 hours on the Jenkins Artifactory service at repo.jenkins-ci.org at 10:00 UTC.

This operation consists in removing all mirrors of the Maven Central artifact repository from repo.jenkins-ci.org to force Maven builds to download mirrored artefacts from the Maven central instead of our Artifactory instance.

This planned brownout is the 2nd tentative related to https://github.com/jenkins-infra/helpdesk/issues/3599 which goal decrease the amount of outbound bandwidth download of our instance.

[Resolved] ldap outage

Tue, 06 Jun 2023 12:45:00 +0000

[Final message] The service has been restored, a misconfiguration was responsible for its outage.

[Initial message] ldap service is down so no authentification working on jira or controllers

[Resolved] infra.ci.jenkins.io cluster migration

Thu, 19 Jan 2023 15:30:00 +0000

[Final message] Migration completed, everything is back to normal.

For users wanting to access infra.ci.jenkins.io, you have to add a new OpenVPN configuration like the one detailed in https://github.com/jenkins-infra/docker-openvpn#howto-get-client-access, but with private.vpn.jenkins.io as remote address instead of vpn.jenkins.io

If you’ve got any problem, please open an help desk issue at https://github.com/jenkins-infra/helpdesk/

[Initial message] In order to migrate infra.ci.jenkins.io to its new cluster (see https://github.com/jenkins-infra/helpdesk/issues/2844), we’ll need to shut down this instance.

Apart from infra.ci.jenkins.io jobs concerning jenkins-infra repositories, jenkins.io preview deployments will also be impacted.

Expected downtime: 1 hour

[Resolved] AKS tempprivatek8s version update

Wed, 26 Oct 2022 14:50:00 +0000

[close issue] Private K8S Upgraded without any problems to kubernetes 1.23.

[open issue] -We’ll do an AKS clusters upgrade from 1.22 to 1.23 this Wednesday, 26th of October 2022, starting at 14:50 UTC.

-See more at https://github.com/jenkins-infra/helpdesk/issues/3053

we expect the following service to get restarted:

infra.ci.jenkins.io

[Resolved] AKS publick8s version update

Thu, 30 Jun 2022 07:00:00 +0000

We’ll do an AKS clusters upgrade from 1.21 to 1.22 this Thursday, 30th of June 2022, starting at 07:00 UTC.

See more at https://github.com/jenkins-infra/helpdesk/issues/2930

No outage is expected, but there could be an impact (1-2 min outage) on the following services if there is an unexpected upgrade issue:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

[Resolved] AKS outage, impacts on infra.ci.jenkins.io and ci.jenkins.io

Tue, 17 May 2022 15:15:00 +0000

[close] AKS outage finished: everything is back to normal.

[update] There is an AKS outage on US East 2, which impacts the Virtual Machines allocation: https://app.azure.com/h/MSR7-9Z0/dffcf0.

[infra.ci.jenkins.io] No builds using Kubernetes-based agent can be executed. “Website Pull Requests preview” feature is unavailable.
- Cause: The Kubernetes cluster temp-privatek8s is unable to launch new Virtual machines nodes: infra.ci.jenkins.io cannot handle builds.
[ci.jenkins.io] Increased build queue as only AWS is used for Docker Agents virtual machines.

[initial message] Cluster automatic scaling to handle builds is not working anymore on infra.ci.jenkins.io so it may have difficulties with builds and it could also impact the preview system used within www.jenkins.io

[Resolved] AKS publick8s version update

Wed, 27 Apr 2022 11:00:00 +0000

We’ll do an AKS clusters upgrade from 1.20 to 1.21 this Wednesday, 27th of April 2022, starting at 11:00 UTC.

See more at https://github.com/jenkins-infra/helpdesk/issues/2866

No outage is expected, but there could be an impact (1-2 min outage) on the following services if there is an unexpected upgrade issue:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

[Resolved] Maintenance Window for Ingress Controllers of Kubernetes Clusters

Fri, 08 Apr 2022 10:30:00 +0000

[Final message]

The maintenance has been completed, all services are operational.

[Initial Message]

A maintenance on the ingress controller of the Azure Kubernetes clusters publick8s and temp-privatek8s is planned for the 08th of April 2022, around 10:30am UTC.

No global outage are expected, but there could be minor impacts (1-2 min outage) on the following services if there is an unexpected upgrade issue:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

Reference: https://github.com/jenkins-infra/helpdesk/issues/2875.

[Resolved] Maintenance operation on infra.ci.jenkins.io + jenkins.io's pull requests

Wed, 30 Mar 2022 13:30:00 +0000

[Initial message] As part of https://github.com/jenkins-infra/helpdesk/issues/2840, the Jenkins infra team will perform a maintenance to infra.ci.jenkins.io.

Impact is for the contributors: the previews on the GitHub pull requests for jenkins.io will fail until the operation is completed.

[Resolved] Maintenance Window on AKS

Tue, 08 Mar 2022 08:00:00 +0000

[Final Message]

Maintenance finished. LDAP was unavailable during restart from 10:10 UTC until 10:15 UTC as per https://github.com/jenkins-infra/helpdesk/issues/2810.

[Initial Message]

A maintenance on the ingress controller of our Azure Kubernetes clusters (publick8s and temp-privatek8s) is planned for the 08th of March at 08:00am UTC.

No global outage are expected, but there could be minor impacts (1-2 min outage) on the following services if there are any unexpected upgrade issues:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

Reference: jenkins-infra/helpdesk#2809

[Resolved] Maintenance Window on AKS

Mon, 20 Dec 2021 11:00:00 +0000

[Final message]

The maintenance has been completed, all services are operational.

[Initial Message]

A maintenance on the ingress controller of the Azure Kubernetes cluster publick8s is planned for the 20th of December, around 11:00am UTC.

No global outage are expected, but there could be minor impacts (1-2 min outage) on the following services if there is an unexpected upgrade issue:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

Reference: https://github.com/jenkins-infra/kubernetes-management/pull/1748.

[Resolved] AKS publick8s version update

Thu, 25 Nov 2021 08:00:00 +0000

INFRA-3118 Notes

[Final Message] The upgrade is finished without any incident, all services are up and running.

Cat tax:

[Initial Message] We’ll do an AKS and EKS clusters upgrade from 1.19 to 1.20 this Thursday, 25th of November 2021, starting at 08:00 UTC.

The service ldap.jenkins.io service might have a downtime due to its PV.

Other services might have a downtime too if their probes take more than one second to respond due to the new timeoutPeriod enforced default value.
Note: the official Jenkins helm chart we’re using has already set the default timeout period to 5 seconds

Apart from that, no outage is expected, but there could be an impact (1-2 min outage) on the following services if there is an unexpected upgrade issue:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

[Resolved] Maintenance Window on AKS

Tue, 26 Oct 2021 08:30:00 +0000

[Final message]

The maintenance has been completed, all services are operational.

[Edit - Monday 25th]

We had to delay the operation as we discovered last minutes items to be changed. Next operation window is planned Tuesday, the 26th of October, at 08:30am UTC.

[Initial Message - Friday 22th]

A maintenance on the ingress controller of the Azure Kubernetes cluster publick8s is planned for the 25th of October, around 12:00am (noon) UTC.

No outage is expected, but there could be an impact (1-2 min outage) on the following service if there is an unexpected upgrade issue:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

Reference: https://github.com/jenkins-infra/charts/pull/1424.