Commit Graph

65 Commits

Author SHA1 Message Date
Thomas Nagy 3f4acd6ff3
Add a new task semaphore system 2018-07-29 12:13:54 +02:00
Thomas Nagy 3fc8c8c68a
Fix deadlocks affecting generated Fortran files #2187 2018-07-28 10:27:54 +02:00
Thomas Nagy 4a71fef6f1
Missing code 2018-03-29 16:49:50 +02:00
Thomas Nagy c80722685c
Apply priorities when consuming tasks - Issue 2137 2018-03-29 11:09:39 +02:00
Thomas Nagy 90c80ea1ae
Apply priorities on dynamically-generated tasks #2137 2018-03-27 20:32:53 +02:00
Thomas Nagy d21446ff71
Update docs for 2018 2018-01-01 20:53:49 +01:00
Matt Selsky c98f8ba2ab Typos 2017-11-28 20:22:10 +01:00
Thomas Nagy 96f4eec8ba Improve the deadlock error message 2017-11-11 10:34:04 +01:00
Thomas Nagy f67d5309f1
Adjust the GAP for tasks from 20*njobs to 5*njobs 2017-10-21 23:21:29 +02:00
Thomas Nagy e68bc5d65b
docs 2017-09-14 22:20:31 +02:00
Thomas Nagy 755a1bf6a5
Simplify task error handling 2017-08-27 10:11:56 +02:00
Thomas Nagy 0659657a8c
Remove old TODO notes 2017-08-25 20:22:46 +02:00
Thomas Nagy 57087471c1
Remove Utils.ex_stack - use traceback.format_exc() 2017-08-19 11:54:58 +02:00
Thomas Nagy 0a1d47c7ec Initialize Parallel.processed at 0 not 1 2017-06-23 13:36:28 +02:00
Thomas Nagy 872052e5c7 Dependencies added after the build starts no longer raise assertions #1982
It appears that quite a few builds use the swig tool technique
of setting build dependencies after the build starts. Missing
entries in Runner/revdeps can make builds non-terminating.
2017-06-20 16:31:08 +02:00
Thomas Nagy f49f162817 Rework the priority system
- Have Task.weight apply to the current task only
- Do not rely on object addresses to set the build order
- Introduce tg.tg_idx_count to count task generators
- Enable propagating/non-propagating weights through Task.tree_weight/Task.weight
2017-06-16 00:14:32 +02:00
Thomas Nagy e5e8397154 Change producer.outstanding.appendleft to producer.outstanding.append 2017-06-14 19:53:01 +02:00
Thomas Nagy 23c0d41aff Schedule tasks using a priority queue 2017-06-14 18:59:28 +02:00
Thomas Nagy ddce7d344e Simplify assertion logic in Runner.py 2017-06-14 14:16:54 +02:00
Thomas Nagy ef75def36a Split incomplete tasks into DAG and non-DAG groups 2017-06-14 13:59:46 +02:00
Thomas Nagy 1d4ce07dea Fix update->extend mismatch in Runner.py - work in progress #1982 2017-06-09 15:54:08 +02:00
Thomas Nagy f75de87cf4 Complete the rename Runner.frozen -> Runner.incomplete #1982 2017-06-08 20:04:24 +02:00
Thomas Nagy 3e1e6586e5 Fix one regression on more_tasks #1981 2017-06-08 14:27:56 +02:00
Thomas Nagy f72167b0e0 Fix one regression on more_tasks #1981 2017-06-07 19:07:27 +02:00
Thomas Nagy 8c32fade86
Improve Task group debugging in Runner.py 2017-04-22 22:09:23 +02:00
Thomas Nagy 14197b713f
Cleanup 2017-03-02 20:43:44 +01:00
Thomas Nagy 8560c18ccb
Process a given task group exactly one 2017-02-20 19:01:33 +01:00
Thomas Nagy 63a53064a3
Provide TaskGroups to replace mem_reducer 2017-02-19 19:58:19 +01:00
Thomas Nagy 497f028a95
Implement a new priority system 2017-02-12 15:29:16 +01:00
Thomas Nagy b7d93c3c4d
Update the year in headers to 2017 2017-02-11 16:13:37 +01:00
Thomas Nagy 5ac8e882e0
Remove the TaskBase class hierarchy level 2017-02-11 15:04:25 +01:00
Thomas Nagy 57b406aaff
Add a task status 'canceled'
The purpose is to skip safely the tasks that have
missing dependencies due to build errors.
2017-01-21 23:47:44 +01:00
Thomas Nagy 4417a3c8c1
Minimize the amount of tasks displayed during the build #1828 2016-10-08 12:53:16 +02:00
Thomas Nagy 445e989594
Unnecessary __slots__ annotation on thread subclass 2016-09-04 16:21:03 +02:00
Thomas Nagy ef6525c0bf
Api docs 2016-06-25 14:49:27 +02:00
Thomas Nagy b59a04a086
Moved the garbage collection code to the object owner 2016-04-28 20:12:20 +02:00
Thomas Nagy a8a918763f
Missing output with -j1 2016-03-03 23:36:18 +01:00
Thomas Nagy 08a39b7b19
Ironpython does not have += on deques 2016-02-28 11:37:00 +01:00
Thomas Nagy 126fc7e4b4
Missing piece of code 2016-02-28 11:30:18 +01:00
Thomas Nagy df956ed5c5
Better task cleanup logic 2016-02-28 10:13:03 +01:00
Thomas Nagy 248d89e85b
Use deques 2016-02-28 10:01:43 +01:00
Thomas Nagy 9dbcc4274b
Fixed the broken logic in process execution 2016-02-27 09:37:04 +01:00
Thomas Nagy 8207a878a9
Enable multiprocessing for faster subprocess execution 2016-02-25 20:03:07 +01:00
Thomas Nagy 4be840bd7a
Reduced Task size (~700B/task) 2016-02-15 00:46:35 +01:00
Thomas Nagy 5a3a89b4de Stop adding incompatible task data into the same dict objects 2015-12-24 18:36:48 +01:00
Thomas Nagy 6c34e2a654
Force scanner functions to run after task failures - Issue 1660 2015-11-20 17:15:33 +01:00
Thomas Nagy 1b34dadd88
Re-added Parallel.error_handler 2014-01-04 11:04:59 +01:00
Thomas Nagy 8ae91879ce
Revert 2f7d561cb0 - there are regressions and the performance improvement is in Pypy only 2014-01-03 22:32:58 +01:00
Thomas Nagy 5032695f26
Simplified the error handling on task status in Runner.py (API change) 2014-01-03 18:22:34 +01:00
Thomas Nagy 2f7d561cb0
Runtime improvements in Runner.py by using deques by default (python >= 2.4) 2014-01-02 23:35:17 +01:00