### UPDATE THESE #####
error_path: /radraid/apps/personal/wasil/ga/debugging_ga_experiments/sm_kidney/error/distributor
######################

### For Condor ###

import:
  simplemind.apt_agents.distributor:
    distributor: CondorDAGDistributor

### UPDATE THESE ###
condor_workpath: /radraid/apps/personal/wasil/ga/debugging_ga_experiments/sm_kidney/distributor/pilot
###################

parallel_n: 5
parallel_sub_n: 5
parallel_misc_n: 5

n_sequential_jobs: 3 

timeout: null

docker: 
    image: "registry.cvib.ucla.edu/sm_release:latest" ### to be updated from simplemind/dev repo

Sections:

[Summary of changes]

• GA Optimization was decoupled from processing and job distribution
  • Now, the Optimizer simply consumes binary strings of parameter sets (and the respective performance metric), and then suggests which of the next parameter sets to try explore next.
  • The Distributor takes these suggested parameter sets and processes them, using whatever job distribution mechanism specified. Parameter sets can be processed even without an optimizer.
• The SM Runner (formerly known as miu) and evaluation were decoupled and made as independent tasks (along with the results compilation task)
  • Each parameter set will undergo a series of Task jobs, specified by the user — by default these are recommended to be (1) running SM Runner (case-wise), (2) running case-wise evaluation, (3) running population-wide results compilation. This can be generalized for the user to define more tasks for their own use.
• Optimization of parameter sets and execution of parameter sets are decoupled.
  • Optimization is done through sm.apt and execution is done through sm.assess
  • The latter does not require any optimization setup.
• Optimization can be run locally within a Docker image (this is the default mode).
• The GA is no longer assumed to be the only optimization method, and can be replaced by overloading the Optimizer class.
• import statements are now reflective of the new simplemind python package (e.g. import simplemind.apt_agents.optimizer )
• Optimization experiments (eg GA experiments) now require 3 configuration files — one for optimizer, one for distributor, and one for tasks.
  • These now incorporate a lot of parameters previously defined in the ga_train_list.csv and the evolve.py entry point.
• The case list for optimization now only requires id image_file reference dataset fields.

[IN HOUSE INSTRUCTIONS]

APT (GA 3.0)

(Steps 1-3 should have been done for you if you haven’t changed your evaluation script)

1. [Optional] Get your evaluation file from in-house QIA and move it to public SM

/src/qia/ga/evaluate_nodule_cg_2D.py

to