mirrors/zig
1
0
Fork 0
mirror of https://codeberg.org/ziglang/zig.git synced 2025-12-06 13:54:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

CI: update_download_page: change docs deployment

Browse source 
The original impetus for making a change here was a typo in --add-header
causing the script to fail. However, upon inspection, I was alarmed that
we were making a --recursive upload to the *root directory* of
ziglang.org. This could result in garbage files being uploaded to the
website, or important files being overwritten. As I addressed this concern,
I decided to take on file compression as well.

Removed compression prior to sending to S3. I am vetoing pre-compressing
objects for the following reasons:
 * It prevents clients from working which do not support gzip encoding.
 * It breaks a premise that objects on S3 are stored 1-to-1 with what is
   on disk.
 * It prevents Cloudflare from using a more efficient encoding, such as
   brotli, which they have started doing recently.

These systems such as Cloudflare or Fastly already do compression on
the fly, and we should interop with these systems instead of fighting them.

Cloudfront has an arbitrary limit of 9.5 MiB for auto-compression. I looked
and did not see a way to increase this limit. The data.js file is currently
16 MiB. In order to fix this problem, we need to do one of the following things:

 * Reduce the size of data.js to less than 9.5 MiB.
 * Figure out how to adjust the Cloudfront settings to increase the max size
   for auto-compressed objects.
 * Migrate to Fastly. Fastly appears to not have this limitation. Note
   that we already plan to migrate to Fastly for the website.
This commit is contained in:
Andrew Kelley 2022-08-31 13:15:42 -07:00
parent 9a0cb34c73
commit c5f1b3224f
1 changed files with 30 additions and 19 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

49
ci/srht/update_download_page
View file

@ -101,32 +101,43 @@ CIDIR="$(pwd)"
cd "$HOME"
# Upload new stdlib autodocs
mkdir -p docs_to_upload/documentation/master/std/
gzip -c -9 "$ZIGDIR/docs/std/index.html" > docs_to_upload/documentation/master/std/index.html
gzip -c -9 "$ZIGDIR/docs/std/data.js" > docs_to_upload/documentation/master/std/data.js
gzip -c -9 "$ZIGDIR/docs/std/main.js" > docs_to_upload/documentation/master/std/main.js
gzip -c -9 "$LANGREF" > docs_to_upload/documentation/master/index.html
$S3CMD put -P --no-mime-magic --recursive --add-header="Content-Encoding:gzip" --add-header="Cache-Control: max-age=0, must-revalidate" "docs_to_upload/" s3://ziglang.org/
# Update autodocs and langref directly to S3 in order to prevent the
# www.ziglang.org git repo from growing too big.
mkdir -p docs_src_to_upload/documentation/master/std/
cp -r "$ZIGDIR/docs/std/src" docs_src_to_upload/documentation/master/std/
$S3CMD put -P --no-mime-magic --recursive --add-header:"Content-Type:text/html" --add-header="Cache-Control: max-age=0, must-revalidate" "docs_src_to_upload/" s3://ziglang.org/
# Please do not edit this script to pre-compress the artifacts before they hit
# S3. This prevents the website from working on browsers that do not support gzip
# encoding. Cloudfront will automatically compress files if they are less than
# 9.5 MiB, and the client advertises itself as capable of decompressing.
# The data.js file is currently 16 MiB. In order to fix this problem, we need to do
# one of the following things:
# * Reduce the size of data.js to less than 9.5 MiB.
# * Figure out how to adjust the Cloudfront settings to increase the max size for
# auto-compressed objects.
# * Migrate to fastly.
$S3CMD put -P --no-mime-magic \
--add-header="Cache-Control: max-age=0, must-revalidate" \
"$LANGREF" s3://ziglang.org/documentation/master/index.html
## Copy without compression:
# mkdir -p docs_to_upload/documentation/master/std/
# cp "$ZIGDIR/docs/std/index.html" docs_to_upload/documentation/master/std/index.html
# cp "$ZIGDIR/docs/std/data.js" docs_to_upload/documentation/master/std/data.js
# cp "$ZIGDIR/docs/std/main.js" docs_to_upload/documentation/master/std/main.js
# cp "$LANGREF" docs_to_upload/documentation/master/index.html
# $S3CMD put -P --no-mime-magic --recursive --add-header="Cache-Control: max-age=0, must-revalidate" "docs_to_upload/" s3://ziglang.org/
$S3CMD put -P --no-mime-magic \
--add-header="Cache-Control: max-age=0, must-revalidate" \
"$ZIGDIR/docs/std/index.html" s3://ziglang.org/documentation/master/std/index.html
$S3CMD put -P --no-mime-magic \
--add-header="Cache-Control: max-age=0, must-revalidate" \
"$ZIGDIR/docs/std/main.js" s3://ziglang.org/documentation/master/std/main.js
$S3CMD put -P --no-mime-magic \
--add-header="Cache-Control: max-age=0, must-revalidate" \
"$ZIGDIR/docs/std/data.js" s3://ziglang.org/documentation/master/std/data.js
$S3CMD put -P --no-mime-magic \
--add-header="cache-control: public, max-age=31536000, immutable" \
"$HOME/$SRC_TARBALL" s3://ziglang.org/builds/
git clone --depth 1 git@github.com:ziglang/www.ziglang.org.git
cd www.ziglang.org
WWWDIR="$(pwd)"
$S3CMD put -P --no-mime-magic --add-header="cache-control: public, max-age=31536000, immutable" "$HOME/$SRC_TARBALL" s3://ziglang.org/builds/
cd "$WWWDIR"
cp "$CIDIR/out/index.json" data/releases.json
git add data/releases.json