Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: reduce API requests in download wheel job [backport 2.16] #11397

Open
wants to merge 1 commit into
base: 2.16
Choose a base branch
from
Open

ci: reduce API requests in download wheel job [backport 2.16] #11397

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 30 additions & 19 deletions .gitlab/download-wheels-from-gh-actions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,37 +6,48 @@ if [ -z "$CI_COMMIT_SHA" ]; then
exit 1
fi

echo "Querying for RUN_ID"

timeout=600 # 10 minutes
start_time=$(date +%s)
end_time=$((start_time + timeout))
# Loop for 10 minutes waiting for run to appear in github
while [ $(date +%s) -lt $end_time ]; do
RUN_ID=$(gh run ls --repo DataDog/dd-trace-py --commit=$CI_COMMIT_SHA --workflow=build_deploy.yml --json databaseId --jq "first (.[]) | .databaseId")
if [ -n "$RUN_ID" ]; then
break;
fi
echo "Waiting for RUN_ID"
sleep 20
done
RUN_ID=$(gh run ls --repo DataDog/dd-trace-py --commit=$CI_COMMIT_SHA --workflow=build_deploy.yml --json databaseId --jq "first (.[]) | .databaseId")
if [ -n "$RUN_ID" ]; then
# The job has not started yet. Give it time to start
sleep 180 # 3 minutes

echo "Querying for RUN_ID"

timeout=600 # 10 minutes
start_time=$(date +%s)
end_time=$((start_time + timeout))
# Loop for 10 minutes waiting for run to appear in github
while [ $(date +%s) -lt $end_time ]; do
RUN_ID=$(gh run ls --repo DataDog/dd-trace-py --commit=$CI_COMMIT_SHA --workflow=build_deploy.yml --json databaseId --jq "first (.[]) | .databaseId")
if [ -n "$RUN_ID" ]; then
break;
fi
echo "Waiting for RUN_ID"
sleep 60
done
fi

if [ -z "$RUN_ID" ]; then
echo "RUN_ID not found"
exit 1
fi

echo "Found RUN_ID: $RUN_ID"
echo "Waiting for workflow to finish"

# wait for run to finish
gh run watch $RUN_ID --interval 45 --exit-status 1 --repo DataDog/dd-trace-py

mkdir pywheels
cd pywheels

echo "Github workflow finished. Downloading wheels"
if [[ $(gh run view $RUN_ID --exit-status --json status --jq .status) != "completed" ]]; then
echo "Waiting for workflow to finish"

# Give time to the job to finish
sleep 300 # 5 minutes

# wait for run to finish
gh run watch $RUN_ID --interval 60 --exit-status 1 --repo DataDog/dd-trace-py
fi

echo "Github workflow finished. Downloading wheels"
# download all wheels
gh run download $RUN_ID --repo DataDog/dd-trace-py

Expand Down
Loading