Publish images to docker hub as soon as possible upon binary release (security)

[mhio]

Problem

The 18.18.2 security release docker images were noticeably behind the 18.18.2 binary releases.

Discussion of some issues and possible solutions appeared in the node repo.

There it was noted that the musl builds support strategy is "experimental" and they will turn up when they turn up, which is one (consistent) component to the delay. Some comments about improving support for musl node were raised and bnoordhuis suggested the image release part should be tracked here.

• 2023-10-13T21:34:43Z There was a note the release was ready at
• 2023-10-14T08:34:19Z About 11 hours for all the required binaries to appear node/docker-node CI/approvals.
• 2023-10-16T15:50:33Z ~66 hours for the docker-library/official-images approvals.
• 2023-10-18T17:54:48Z Image tags turn up in the docker registry for the 18.18.2 tag specifically the last_pushed dates are from 2023-10-18T17:54:48.75523Z to 2023-10-18T20:30:39.567863Z

I believe the other issue in this case was approvals on docker-library/official-images over a weekend.

Solution

Not sure exactly, and this is probably only of importance for high severity security releases. This issue is more for discussion.

One thought was to structure the image release CI/approvals as per the supported platforms list so the Tier 1/Tier 2 supported platforms appear earlier. But that would only be a small improvement, still with the substantial delay to build the images. I could imagine a worst case where something in the experimental builds does fail which would delay everything which would be nice to avoid.

Alternatives to Consider

To discuss.

[mhio]

These are the timestamps on the binary distribution sites. The times don't line up with the github notes, maybe they are US West times?

https://nodejs.org/dist/v18.18.2/

node-v18.18.2-linux-x64.tar.gz                     13-Oct-2023 14:02            44553491
node-v18.18.2-linux-x64.tar.xz                     13-Oct-2023 14:03            23875932
node-v18.18.2-linux-armv7l.tar.gz                  13-Oct-2023 14:04            41120209
node-v18.18.2-linux-armv7l.tar.xz                  13-Oct-2023 14:05            20932900
node-v18.18.2-linux-s390x.tar.gz                   13-Oct-2023 14:09            44805592
node-v18.18.2-linux-s390x.tar.xz                   13-Oct-2023 14:11            22707508
node-v18.18.2-linux-ppc64le.tar.gz                 13-Oct-2023 14:23            46561105
node-v18.18.2-linux-ppc64le.tar.xz                 13-Oct-2023 14:25            24287180
node-v18.18.2.pkg                                  13-Oct-2023 14:42            71187652
node-v18.18.2.tar.gz                               13-Oct-2023 14:43            86108679
node-v18.18.2.tar.xz                               13-Oct-2023 14:47            40834428
node-v18.18.2-headers.tar.gz                       13-Oct-2023 14:51             8713368
node-v18.18.2-headers.tar.xz                       13-Oct-2023 14:51              479428
node-v18.18.2-linux-arm64.tar.gz                   13-Oct-2023 18:03            44407009
node-v18.18.2-linux-arm64.tar.xz                   13-Oct-2023 18:05            23144660

https://unofficial-builds.nodejs.org/download/release/v18.18.2/

node-v18.18.2-headers.tar.gz                       14-Oct-2023 02:52             8713368
node-v18.18.2-headers.tar.xz                       14-Oct-2023 02:52              479428
node-v18.18.2-linux-x64-musl.tar.gz                14-Oct-2023 03:34            45507211
node-v18.18.2-linux-x64-musl.tar.xz                14-Oct-2023 03:37            24607896
node-v18.18.2-linux-armv6l.tar.gz                  14-Oct-2023 04:39            41243769
node-v18.18.2-linux-armv6l.tar.xz                  14-Oct-2023 04:40            21040624

[pierceray]

What is the expected turnaround time for a new LTS release of the docker-node images? The lts/iron 20.9.0 was released this morning.

[shaneog]

@pierceray That's the main problem here; there are no musl builds yet for the new versions, and so no images can be produced since this repo requires both official and unofficial (musl) builds to be available before new Docker images are produced.

That seems to be the root of @mhio's issue (and mine).

[pierceray]

Thank you for the explanation.

[pierceray]

It looks like those musl builds exist now.