Make CI faster (#2662)

Right now the CI takes ~30 minutes; the goal is to have it ending in
< 15 min.
The basic trick is to run the longer jobs (no_x86_64 and masan) only
with the recently updated pcaps. The same jobs will run again on schedule
(every night) testing all the traces.

This way the CI will be "green" (hopefully!) earlier while pushing new
commit/PR; full tests are simply delayed.

Details: when `NDPI_TEST_ONLY_RECENTLY_UPDATED_PCAPS` is set,
`tests/do.sh` checks only the latest 10 pcaps (i.e. the more recent pcap
added/updated) for *every* configuration.

Notes that no_x86_64 and masan jobs run twice: when pushing/merging and
on schedule (every night)
This commit is contained in:
Ivan Nardi 2025-01-11 19:09:48 +01:00 committed by GitHub
parent 957a050500
commit 655360728b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 71 additions and 16 deletions

47
.github/workflows/build_masan.yml vendored Normal file
View file

@ -0,0 +1,47 @@
name: Memory Sanitizer
on:
push:
branches:
- dev
pull_request:
branches:
- dev
types: [opened, synchronize, reopened]
release:
types: [created]
workflow_dispatch:
schedule:
#At the end of every day
- cron: '0 0 * * *'
jobs:
memorysanitizer:
name: masan (ubuntu-22.04)
runs-on: ubuntu-22.04 # TODO: some issues with masan/clang/ubuntu-24.04
env:
CC: clang
CFLAGS: -Wextra -Werror -DNDPI_EXTENDED_SANITY_CHECKS
# Idea to have significant faster tests while pushing/merging: test all pcaps only on schedule runs; otherwise tests only the pcaps updated/added recently
NDPI_TEST_ONLY_RECENTLY_UPDATED_PCAPS: ${{ (github.event_name == 'schedule') && '0' || '1' }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison libjson-c-dev libpcap-dev rrdtool librrd-dev parallel libpcre3-dev libmaxminddb-dev
- name: Configure nDPI
run: |
./autogen.sh --enable-option-checking=fatal --with-pcre2 --with-maxminddb --with-memory-sanitizer --disable-memory-track-origins
- name: Build nDPI
run: |
make -j $(nproc) all
make -j $(nproc) -C example ndpiSimpleIntegration
make -j $(nproc) -C rrdtool
- name: Tests
run: |
NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 ./tests/do.sh
# Some issues with masan + json-c. Disable the test as workaround
# ./tests/do-unit.sh
./tests/do-dga.sh