Use --parallel 4 when running tests on CI, 6 in precommit#2666
Use --parallel 4 when running tests on CI, 6 in precommit#2666karyon wants to merge 2 commits intoe-valuation:mainfrom
Conversation
On my machine, wall clock time for "./manage.py run test" drops from 43 to 20s with 6 threads. More threads increased the overhead for creating databases more than it reduced the time for the actual tests. The github runners only have 4 cores.
karyon
force-pushed
the
karyon/parallel-tests
branch
from
56f709a to
75696fc
Compare
karyon
force-pushed
the
karyon/parallel-tests
branch
from
374970c to
e228804
Compare
|
|
||
| - name: Run tests | ||
| run: python manage.py test --shuffle | ||
| run: python manage.py test --shuffle --parallel 4 |
There was a problem hiding this comment.
Django uses the number of physical CPU cores when you just specify --parallel, or --parallel auto (docs)
I think it would make sense to just use the auto detection instead of hardcoding the 4 and the 6 here. For 4, you said that this is equal to the number of cores. If GitHub runners change in the future, with "auto", everything keeps working optimally.
For the dev environment, we have people running in different setups, so I'd also say that using the core count is a better approximation. How many cores does your system have, 8? How much slower was it with 8 threads compared to 6?
There was a problem hiding this comment.
Definitely agree for CI, for dev I am not so sure, don't want any surface laptops to go up in flames :D Maybe cpucount/2 or so?
There was a problem hiding this comment.
Django can only do "auto", anything else would require code from us. I think "auto" will be fine, even for the weaker laptops. Note that this is only precommit. For test, we don't change the default.
There was a problem hiding this comment.
Are we going to regret this when trying to reproduce CI failures locally? It says in the docs that
Django distributes test cases — unittest.TestCase subclasses — to subprocesses. If there are fewer test case classes than configured processes, Django will reduce the number of processes accordingly.
but does that mean if I know the shuffle seed and use parallel=4 that I will get the same test order?
3 participants
On my machine, wall clock time for "./manage.py run test" drops from 43 to 20s with 6 threads. More threads increased the overhead for creating databases more than it reduced the time for the actual tests. The github runners only have 4 cores.