Get.jenkins.io on Jenkins Status Page

[Resolved] Brownout (temporary disruption) of TUNA mirror

Mon, 07 Sep 2026 13:00:00 +0000

Final message

TUNA mirror has been permanently removed. See https://github.com/jenkins-infra/helpdesk/issues/5307 for the next steps.

Initial message

The “TUNA” (mirrors.tuna.tsinghua.edu.cn) mirror for Jenkins downloads (get.jenkins.io) is temporarily disabled to study redirections behaviors.

Please report any download issue from Asia in https://github.com/jenkins-infra/helpdesk/issues/5255.

More details in https://github.com/jenkins-infra/helpdesk/issues/5255#issuecomment-5571379729.

[Resolved] [publick8s cluster] Azure publick8s Upgrade to Kubernetes 1.33.5

Mon, 15 Dec 2025 08:30:00 +0000

[Final Message] upgrade went well, no impact on production.

[Initial Message] Monday 15 December 2025 at 08:30PM UTC, we’ll upgrade the AKS(Azure) cluster publick8s to Kubernetes 1.33.5. It should take between 60 and 120 min.

The expected impacts are:

Restart of the following services, with a downtime between 2 and 15 min (depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-in/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) on:

get.jenkins.io (mirror redirector)
issue-redirect.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
plugin-health.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

More details in https://github.com/jenkins-infra/helpdesk/issues/4820#issuecomment-3548729217.

[Resolved] Outage on get.jenkins.io and updates.jenkins.io

Thu, 09 Oct 2025 07:08:20 +0000

[Final Message] The outage seems resolved https://github.com/jenkins-infra/helpdesk/issues/4827

[Initial Message] we are encountering an outage on get.jenkins.io and updates.jenkins.io our publick8s cluster. We are seeing problems that azure doesn’t acknowledge, probably related to an Azure outage.

see https://github.com/jenkins-infra/helpdesk/issues/4827 for information

[Resolved] Outage on get.jenkins.io and updates.jenkins.io

Tue, 07 Oct 2025 07:03:00 +0000

[Final Message] The outage seems resolved https://github.com/jenkins-infra/helpdesk/issues/4827 . We did a refresh of the 3 nodes of publick8s but were impacted with an azure outage that is not public yet.

[Initial Message] We are encountering an outage on get.jenkins.io, currently investigating. The same problem is visible on updates.jenkins.io.

[Resolved] Azure cluster publick8s migrated to a new location and upgraded to Kubernetes 1.32

Thu, 25 Sep 2025 11:30:00 +0000

[Final Message] Outage finished on Friday 26 September 2025 at 09:35am UTC and post mortem published: https://github.com/jenkins-infra/helpdesk/issues/4813#issuecomment-3339518665.

[Update] Friday 26 September 2025 at 09:35am UTC: ongoing production outage on the cluster (unexpected side effect of the migration): https://github.com/jenkins-infra/helpdesk/issues/4813.

[Update] Migration was finished on Thursday 25 September 2025 at 03:30PM UTC. See https://github.com/jenkins-infra/helpdesk/issues/4617#issuecomment-3334826339.

[Initial Message] Thursday 25 September 2025, starting at 11:30AM UTC, we’ll perform a migration + upgrade of the AKS(Azure) cluster publick8s to a new location and to Kubernetes 1.32.

The expected impacts are:

Restart of the following services, with a downtime between 2 and 15 min (depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-in/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) on:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
plugin-health.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

More details in https://github.com/jenkins-infra/helpdesk/issues/4617.

[Resolved] [publick8s cluster] Azure publick8s Upgrade to Kubernetes 1.31.6

Thu, 24 Apr 2025 07:30:00 +0000

[Final Message] upgrade went well, no impact on production.

[Initial Message] Thursday 24 April 2025, at 07:30PM UTC, we’ll upgrade the AKS(Azure) cluster publick8s to Kubernetes 1.31.6. It should take between 60 and 120 min.

The expected impacts are:

⚠️ Last Kubernetes upgrade (to 1.30) we saw an outage on update-center (updates.jenkins.io) for a few seconds !

Restart of the following services, with a downtime between 2 and 15 min (depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-in/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) on:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
plugin-health.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

More details in https://github.com/jenkins-infra/helpdesk/issues/4546#issuecomment-2674545933.

[Resolved] [publick8s cluster] Azure publick8s Upgrade to Kubernetes 1.30.7

Thu, 20 Feb 2025 13:30:00 +0000

[Final Message] upgrade went well but we saw an outage on update center for about 2 to 3 mn we will analyse further the logs to try to understand were it’s coming from

[Initial Message] Thurday 20 February 2025, at 01:30PM UTC, we’ll upgrade the AKS(Azure) cluster publick8s to Kubernetes 1.30.7. It should take between 60 and 120 min.

The expected impacts are:

Restart of the following services, with a downtime between 2 and 15 min (depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-in/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) on:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
plugin-health.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

More details in https://github.com/jenkins-infra/helpdesk/issues/4258#issuecomment-2663553626.

[Resolved] AKS cluster upgrade to 1.29.10 (patch)

Wed, 18 Dec 2024 09:45:00 +0000

We’re performing AKS cluster upgrade from 1.29.7 to 1.29.10 the 18 December 2024, starting at 09:45 UTC.

The expected impacts are:

Restart of the following services, with an outage (between 2 to 15 min: depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-In/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

[Resolved] Disruption of mirror selections on get.jenkins.io and Update Center

Thu, 05 Dec 2024 13:00:00 +0000

User reported that mirror selection is acting randomly with message in mirrorbit web UI saying “Cannot locate the client IP (ASN0)”.

Requests are still served but it may disrupt download performances.

More details in https://github.com/jenkins-infra/helpdesk/issues/4441.

[Resolved] [get.jenkins.io] Reset mirrors database

Wed, 16 Oct 2024 12:45:00 +0000

Thursday October 30 2024, starting at 12:45 UTC, the Jenkins download website (get.jenkins.io/mirrors.jenkins.io) will be disrupted for a technical operation on its database.

The goal is to solve the HTTP/404 errors on old files caught in the following issue: https://github.com/jenkins-infra/helpdesk/issues/4330.

The Jenkins Infra team does not expect and outage but slower download as all traffic will be sent to the fallback system until mirrors are added back and fully scanned.

More details in https://github.com/jenkins-infra/helpdesk/issues/4330.

[Resolved] Update Azure AKS cluster `publick8s` to 1.29

Fri, 23 Aug 2024 09:00:00 +0000

Friday 23 August at 07:30am UTC, the AKS cluster publick8s will be upgraded to Kubernetes 1.29.7. It should take between 30 and 60 min.

The expected impacts are:

Restart of the following services, with a downtime between 2 and 15 min (depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-in/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) on:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
plugin-health.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

More details in https://github.com/jenkins-infra/helpdesk/issues/4161.

[Resolved] Update azure clusters publick8s Kubernetes version to 1.28

Thu, 04 Jul 2024 06:30:00 +0000

[Final Message] The operation finished successfully at 7:15am UTC.

More details on https://github.com/jenkins-infra/helpdesk/issues/4144#issuecomment-2196274746.

[Initial message] We’re doing an publick8s AZ cluster upgrade from 1.27 to 1.28 tomorrow, 4th of July 2024, starting at 6:30 UTC (8h30 paris time, 12h00 IST).

The expected impacts are:

Restart of the following service, with an outage (between 2 to 15 min: depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-In/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) due to public IP metadata changes and falco + Nginx upgrades:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
plugin-health.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

[Resolved] Update azure clusters publick8s Kubernetes version to 1.27

Tue, 19 Mar 2024 08:30:00 +0000

[Final Message] The operation finished successfully at 9:35am UTC.

More details on https://github.com/jenkins-infra/helpdesk/issues/3948#issuecomment-1991181964.

[Initial message] We’re doing an publick8s AZ cluster upgrade from 1.26 to 1.27 today 19th of Mars 2024, starting at 8:30 UTC (9h30 paris time).

The expected impacts are:

Restart of the following service, with an outage (between 2 to 15 min: depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-In/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) due to public IP metadata changes and falco + Nginx upgrades:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

[Resolved] get.jenkins.io latest 2.443 issue

Tue, 30 Jan 2024 15:00:00 +0000

[Final Message]

Release Core packaging process finished with success, both services are back to normal.

[Initial Message]

Tuesday January 30 2024, starting at 15:05 UTC, get.jenkins.io and updates.jenkins.io return an error when trying to download the latest Jenkins Weekly release 2.443, due to an issue in the release process.

More details in https://github.com/jenkins-infra/helpdesk/issues/3927

[Resolved] get(mirrors).jenkins.io storage migration

Fri, 26 Jan 2024 12:00:00 +0000

A maintenance on get.jenkins.io (mirrors.jenkins*) will be performed with a partial service disruption the Friday 26 January 2024 starting at 12:00pm (noon time) UTC.

The goal of this maintenance is to migrate the underlying storage to a new system in Azure.

See https://github.com/jenkins-infra/helpdesk/issues/3917 for more details.

[Resolved] get.jenkins.io maintenance

Mon, 08 Jan 2024 15:15:00 +0000

[Final Message] THe migration is finally finished at noon UTC the 09th. There were 2 downtimes:

2 min the 8th around 17:30 UTC (TLS errors) due to a missing ingress configuration
5 min the 9th around 11:50 UTC (HTTP/500 errors) due to a race condition in helm chart deployments

[Initial Message] Monday January 8 2024, starting at 15:15 UTC, we’ll proceed to a maintenance on the service get.jenkins.io hosted in the publick8s Azure Kubernetes cluster.

The following services will be slower to respond or even might be disrupted:

get.jenkins.io (Jenkins download redirection to mirrors)
mirrors.jenkins-ci.org (Jenkins download redirection to mirrors)
mirrors.jenkins.io (Jenkins download redirection to mirrors)
fallback.get.jenkins.io

The goal is to change the helm chart use to deploy the service from the legacy mirrorbits to the new mirrorbits-parent to benefit from:

Apache running in arm64 Linux systems (cheaper)
Readonly filesystem for shared data (safety)
Same features as the current updates-jenkins-io service (maintenance and sustainability)

More details in: https://github.com/jenkins-infra/helpdesk/issues/3828

[Resolved] Update publick8s AKS cluster to Kubernetes 1.26

Thu, 09 Nov 2023 08:30:00 +0000

[Final Message] The operation finished successfully at 11:00am UTC. The monitoring showed an outage of 6 minutes on the LDAP and 2 minutes on the incremental publisher.

More details on https://github.com/jenkins-infra/helpdesk/issues/3683#issuecomment-1803566680.

[Initial Message] The AKS cluster publick8s will be upgraded from Kubernetes 1.25 to 1.26 the Thursday 09th of November 2023, starting at 08:30am UTC.

The expected impacts are:

Restart of the following service, with an outage (between 2 to 15 min: depends on Azure Storage systems):

ldap.jenkins.io
weekly.ci.jenkins.io
incrementals.jenkins.io

Temporary disruptions (no Log-In/log-out possible during the LDAP restart) on:

accounts.jenkins.io
ci.jenkins.io, trusted.ci.jenkins.io, infra.ci.jenkins.io, weekly.ci.jenkins.io, release.ci.jenkins.io, cert.ci.jenkins.io
issues.jenkins.io (JIRA)

Temporary disruptions (slow requests or outage of a few minutes) due to public IP metadata changes and falco + Nginx upgrades:

get.jenkins.io (mirror redirector)
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
repo.azure.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

[Resolved] arm64 migration of mirrorbits

Wed, 25 Oct 2023 16:00:00 +0000

[Final message]

The operation is postponed as we encountered issues with some binaries not built for arm64 architecture in our images, no interruption of services.

[Initial message]

Wednesday October 25 2023, starting at 16:00 UTC, we’ll proceed to the arm64 migration of the mirrorbits service responsible for get.jenkins.io, fallback.get.jenkins.io, mirrors.jenkins.io and mirror.jenkins-ci.org.

There might be a brief interruption of these services.

Related help desk issue: https://github.com/jenkins-infra/helpdesk/issues/3619

[Resolved] arm64 migration of public nginx ingress on publick8s cluster

Tue, 10 Oct 2023 12:00:00 +0000

[Final message]

The operation is now finished, no interruption of services.

[Initial message]

Thursday October 19 2023, starting at 12:00 UTC, we’ll proceed to the arm64 migration of the publick8s nginx ingress, hosting most of Jenkins Infrastructure public-facing services in order to reduce costs.

There might be a brief interruption of these services.

Related help desk issue: https://github.com/jenkins-infra/helpdesk/issues/3619

[Resolved] Update AKS clusters Kubernetes version to 1.25

Fri, 07 Jul 2023 08:30:00 +0000

[Final Message] The operation finished at 15:30 UTC after a full downtime of ~3 hours on the publick8s cluster due to a network issue.

More details in https://github.com/jenkins-infra/helpdesk/issues/3582.

[Initial Message] We’re doing an AKS clusters upgrade from 1.24 to 1.25 the Friday 07th of July 2023, starting at 08:30 UTC.

No outage is expected, but there will be application restarts, including a downtime between 3 to 10 minutes for ldap.jenkins.io.

Other services potentially impacted:

accounts.jenkins.io
get.jenkins.io
incrementals.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
previews of *.jenkins.io pull requests (infra.ci.jenkins.io)
release.ci.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

[Resolved] mirrorbits migration

Mon, 12 Jun 2023 12:00:00 +0000

[Final message] Migration completed.

[Initial message] A migration of the “mirrorbits” service from prodpublick8s cluster to publick8s cluster is planned today on Monday 12st of June 2023 at 12h00 UTC. While there should not be any interruption, the corresponding service may be down for less than one hour.

More details on https://github.com/jenkins-infra/helpdesk/issues/3351#issuecomment-1586969797.

[Resolved] Incident on get.jenkins.io (Mirror Redirector)

Wed, 19 Apr 2023 07:32:00 +0000

[Update] Triggering a node pool image OS upgrade to recycle volume mounts. Expect a 4-5 minutes outage on the LDAP connection (e.g. authentication on issues.jenkins.io and (.*)ci.jenkins.io)

[Initial message] The mirror redirector service (get.jenkins.io) is slow and may responds HTTP/502 errors on some requests as reported by a user at 7:32am UTC - https://github.com/jenkins-infra/helpdesk/issues/3525.

Investigation in progress, this page will be updated accordingly to our findings.

[Resolved] Update Azure Service Principal credential for the AKS cluster prodpublick8s

Tue, 07 Mar 2023 13:00:00 +0000

[Closing] The operation went well and took 30 minutes, including 5 minute of restart for ldap.jenkins.io.

[Opening] We’ll perform an operation on the AKS cluster prodpublick8s the 07 of March 2023 at 13:00 UTC.

The goal of this operation is to rotate the (soon-to-be-expired) Azure Service Principal credential for this cluster.

This operation will restart all the machines of the cluster.

The following services will be restarted (e.g. unavailable during the restart phase):

ldap.jenkins.io (⚠️ authentication on ci.jenkins.io, issues.jenkins.io and repo.jenkins-ci.org is affected)
reports.jenkins.io

The following services will be migrated without downtime (but slower response time during the operation):

accounts.jenkins.io
get.jenkins.io
incrementals.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
plugins.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

See more at https://github.com/jenkins-infra/helpdesk/issues/3433.

[Resolved] Update AKS clusters Kubernetes version to 1.24

Thu, 23 Feb 2023 10:30:00 +0000

We’re doing an AKS clusters upgrade from 1.23 to 1.24 today, Thursday 23th of February 2023, starting at 10:30 UTC.

See more at https://github.com/jenkins-infra/helpdesk/issues/3387

No outage is expected, but there could be an impact on the Azure artifact caching proxy. In the case it’s not responding, the builds will simply fallback to repo.jenkins-ci.org to retrieve artifacts.

Other services potentially impacted:

accounts.jenkins.io
get.jenkins.io
incrementals.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
previews of *.jenkins.io pull requests (infra.ci.jenkins.io)
release.ci.jenkins.io
reports.jenkins.io
stories.jenkins.io
uplink.jenkins.io
www.jenkins.io

[Resolved] AKS publick8s version update

Thu, 27 Oct 2022 08:00:00 +0000

[close] Upgrade done, no issue or problems to report. see more at https://github.com/jenkins-infra/helpdesk/issues/3053

[open] We’ll do an AKS clusters upgrade from 1.22 to 1.23 this Thursday, 27th of October 2022, starting at 08:00 UTC.

See more at https://github.com/jenkins-infra/helpdesk/issues/3053

No outage is expected, but there could be an impact (1-2 min outage) on the following services if there is an unexpected upgrade issue:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

[Resolved] AKS publick8s version update

Thu, 30 Jun 2022 07:00:00 +0000

We’ll do an AKS clusters upgrade from 1.21 to 1.22 this Thursday, 30th of June 2022, starting at 07:00 UTC.

See more at https://github.com/jenkins-infra/helpdesk/issues/2930

No outage is expected, but there could be an impact (1-2 min outage) on the following services if there is an unexpected upgrade issue:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

[Resolved] AKS publick8s version update

Wed, 27 Apr 2022 11:00:00 +0000

We’ll do an AKS clusters upgrade from 1.20 to 1.21 this Wednesday, 27th of April 2022, starting at 11:00 UTC.

See more at https://github.com/jenkins-infra/helpdesk/issues/2866

No outage is expected, but there could be an impact (1-2 min outage) on the following services if there is an unexpected upgrade issue:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

[Resolved] Maintenance Window for Ingress Controllers of Kubernetes Clusters

Fri, 08 Apr 2022 10:30:00 +0000

[Final message]

The maintenance has been completed, all services are operational.

[Initial Message]

A maintenance on the ingress controller of the Azure Kubernetes clusters publick8s and temp-privatek8s is planned for the 08th of April 2022, around 10:30am UTC.

No global outage are expected, but there could be minor impacts (1-2 min outage) on the following services if there is an unexpected upgrade issue:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

Reference: https://github.com/jenkins-infra/helpdesk/issues/2875.

[Resolved] Maintenance Window on AKS

Tue, 08 Mar 2022 08:00:00 +0000

[Final Message]

Maintenance finished. LDAP was unavailable during restart from 10:10 UTC until 10:15 UTC as per https://github.com/jenkins-infra/helpdesk/issues/2810.

[Initial Message]

A maintenance on the ingress controller of our Azure Kubernetes clusters (publick8s and temp-privatek8s) is planned for the 08th of March at 08:00am UTC.

No global outage are expected, but there could be minor impacts (1-2 min outage) on the following services if there are any unexpected upgrade issues:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

Reference: jenkins-infra/helpdesk#2809

[Resolved] Maintenance Window on AKS

Mon, 20 Dec 2021 11:00:00 +0000

[Final message]

The maintenance has been completed, all services are operational.

[Initial Message]

A maintenance on the ingress controller of the Azure Kubernetes cluster publick8s is planned for the 20th of December, around 11:00am UTC.

No global outage are expected, but there could be minor impacts (1-2 min outage) on the following services if there is an unexpected upgrade issue:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

Reference: https://github.com/jenkins-infra/kubernetes-management/pull/1748.

[Resolved] AKS publick8s version update

Thu, 25 Nov 2021 08:00:00 +0000

INFRA-3118 Notes

[Final Message] The upgrade is finished without any incident, all services are up and running.

Cat tax:

[Initial Message] We’ll do an AKS and EKS clusters upgrade from 1.19 to 1.20 this Thursday, 25th of November 2021, starting at 08:00 UTC.

The service ldap.jenkins.io service might have a downtime due to its PV.

Other services might have a downtime too if their probes take more than one second to respond due to the new timeoutPeriod enforced default value.
Note: the official Jenkins helm chart we’re using has already set the default timeout period to 5 seconds

Apart from that, no outage is expected, but there could be an impact (1-2 min outage) on the following services if there is an unexpected upgrade issue:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

[Resolved] Maintenance Window on AKS

Tue, 26 Oct 2021 08:30:00 +0000

[Final message]

The maintenance has been completed, all services are operational.

[Edit - Monday 25th]

We had to delay the operation as we discovered last minutes items to be changed. Next operation window is planned Tuesday, the 26th of October, at 08:30am UTC.

[Initial Message - Friday 22th]

A maintenance on the ingress controller of the Azure Kubernetes cluster publick8s is planned for the 25th of October, around 12:00am (noon) UTC.

No outage is expected, but there could be an impact (1-2 min outage) on the following service if there is an unexpected upgrade issue:

accounts.jenkins.io
beta.accounts.jenkins.io
customize.jenkins.io
get.jenkins.io
incrementals.jenkins.io
infra.ci.jenkins.io
javadoc.jenkins.io
jenkins-wiki-exporter.jenkins.io
ldap.jenkins.io
plugins.jenkins.io
release.ci.jenkins.io
reports.jenkins.io
www.jenkins.io
uplink.jenkins.io

Reference: https://github.com/jenkins-infra/charts/pull/1424.

[Resolved] Windows LTS Package Unavailable

Wed, 02 Jun 2021 18:59:00 +0000

Due to a technical issue in the release process, the Windows package for the LTS Jenkins release was unavailable during almost 13 hours.

The release process has been fixed and executed successfully the 03rd of June 2021 at 06:50 UTC and finished at 07:02 UTC.

At 07:00 UTC, the Windows package was available again from https://jenkins.io/download.

[Resolved] Windows Weekly Package Unavailable

Tue, 25 May 2021 11:30:00 +0000

Due to a technical issue in the release process, the Windows package for the weekly Jenkins release was unavailable from 2021-05-25 11:30 UTC until 2021-05-26 07:51 UTC.

A Post-Mortem is available on hackmd.io.

[Resolved] get.jenkins.io disk quota exceeded

Mon, 19 Apr 2021 18:50:00 +0000

Issue 4270 Infra mailing list NOTES

The get.jenkins.io and fallback.get.jenkins.io pods are unable to mount their Azure file storage. The mount failure prevents the files from being served by get.jenkins.io.

2021-04-19T21:12:00-00:00

The disk quota issue has been solved using Microsoft documentation So mirrorbits containers are back to normal. We tested that service was operational using https://fallback.get.jenkins.io At the moment, the traffic is still routed to pkg.origin.jenkins.io but it has no impact on the availability of the service. We will switch back the traffic at a better time.

We validated that the service was operational using this mirrorlist

[Resolved] get.jenkins.io dns update

Sun, 13 Dec 2020 13:20:00 +0000

INFRA-2846

Because of the previous get.jenkins.io outage, we temporarily redeployed the service on a virtual machine. The azure file storage issue is now resolved and the service ready to handle the traffic as it used to be. I am going to update the dns record to redirect from the temporary virtual machine to our kubernetes cluster.

No downtime is expected

[Resolved] get.jenkins.io issues

Thu, 12 Nov 2020 15:54:00 +0000

2020-11-23: Azure is investigating the root cause of the issue
2020-11-19: The Azure blog storage is back without additional information
*2020-11-14: We temporarily deploy get.jenkins.io outside Azure
2020-11-17: We have a call with Azure support today

The network volume used by get.jenkins.io is currently down. This outage affect every services relying on get.jenkins.io like updates.jenkins.io and pkg.jenkins.io

Twitter Discussion