Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically file bugs if BPF performance changes by more than 2σ #3981

Open
1 of 4 tasks
Alan-Jowett opened this issue Nov 3, 2024 · 0 comments
Open
1 of 4 tasks
Assignees
Labels
P2 tests triaged Discussed in a triage meeting
Milestone

Comments

@Alan-Jowett
Copy link
Member

  • Create script to get list of metrics whose most recent value is more than 2σ above the 30 days run mean value. See: check_results
  • Create a scheduled workflow that executes the query.
  • For each metric outside the range, either file a new issue or update the currently active issue.
  • If the most recent run is more then 3 days old, file an new or update an existing issue.

This project has a problem that the metrics on Perf Dashboard are not being monitored for changes.

There should be issues automatically created for any metric that deviates by more than 2σ from the last 30 run mean value.

The script already exists in the bpf_conformance repo and requires a scheduled workflow to execute it. This should be a separate workflow from the existing one to ensure that it always runs, to catch cases where the existing workflow isn't running correclty.

@shankarseal shankarseal added P2 triaged Discussed in a triage meeting tests labels Nov 4, 2024
@shankarseal shankarseal added this to the 2411 milestone Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 tests triaged Discussed in a triage meeting
Projects
None yet
Development

No branches or pull requests

3 participants