# changelog

author: Carl Edquist <edquist@cs.wisc.edu> - 1.18.44
changelog:
  - Add Slurm SystemCpu accounting when job completes (SOFTWARE-3825)
  - Add support for decimal seconds fields from sacct (SOFTWARE-3826)
  - Pull in various updates to slurm scripts from htcondor's blap (SOFTWARE-3587)
  - Move local submit attributes scripts from spec to sources (SOFTWARE-3749)


author: Carl Edquist <edquist@cs.wisc.edu> - 1.18.46
changelog:
  - Fix an issue where the slurm binpath always returned scontrol (SOFTWARE-3986)
  - Python 3 compatibility (#7)
  - Handle extra-quoted arguments to condor_submit.sh (SOFTWARE-3993)
  - Expand env vars in configured slurm_binpath (#10)
  - Fix cluster handling in slurm_status.py (#13)
  - Introduce blah_job_env_confs for dynamic env var expansion (SOFTWARE-2521)
  - Add EXIT trap to remove barrier file (SOFTWARE-3930)
  - Amended file credits for Matt Farrellee (#17)


author: Brian Lin <blin@cs.wisc.edu> - 1.18.48-1
changelog:
  - Add an option for HTCondor batch system to expand references to $HOME in the job environment (#23)
  - Add efficient querying of LSF job statuses (#14)
  - Improve logic when proxy refresh is disabled (#24)
  - Fix invalid argument, output, and error format for HTCondor job submission (#22)


author: Brian Lin <blin@cs.wisc.edu> 2.0.0-1
changelog:
  - Add GPU support (#12)
  - Improve multi-node support for Slurm (HTCONDOR-166)
  - More efficient status queries for Slurm (HTCONDOR-104)
  - More robust job ID extraction from SLurm (HTCONDOR-259)
  - Better interfacing with PBS and Slurm
  - More Python 3 fixes for PBS and Slurm scripts
  - Convert LSF script to Python 3 (HTCONDOR-333, #34)
  - Add Debian build files


author: Tim Theisen <tim@cs.wisc.edu> 2.0.1-1
changelog:
  - Convert to pure Python 3 (no Python 2 compatibility)
  - Restore blah_libexec_directory to the blah.config file


author: Tim Theisen <tim@cs.wisc.edu> 2.1.0-1
changelog:
  - Fix bug where GPU request was not passed onto the batch script
  - Fix problem where proxy symlinks were not cleaned up by not creating them
  - Fix bug where output files are overwritten if no transfer output remap
  - Added support for passing in extra submit arguments from the job ad


author: Tim Theisen <tim@cs.wisc.edu> 2.1.1-1
changelog:
  - Add Python 2 support back for Enterprise Linux 7
  - Allow the user to override system configuration files
  - Enable flexible configuration via a configuration directory
  - Fix Slurm resource usage reporting


author: Tim Theisen <tim@cs.wisc.edu> - 8.9.8-1
changelog:
  - Added htcondor.dags and htcondor.htchirp to the HTCondor Python bindings
  - New condor_watch_q tool that efficiently provides live job status updates
  - Added support for marking a GPU offline while other jobs continue
  - The condor_master command does not return until it is fully started
  - Deprecated several Python interfaces in the Python bindings


author: Tim Theisen <tim@cs.wisc.edu> - 8.8.11-1
changelog:
  - HTCondor now properly tracks usage over vanilla universe checkpoints
  - New ClassAd equality and inequality operators in the Python bindings
  - Fixed a bug where removing in-use routes could crash the job router
  - Fixed a bug where condor_chirp would abort after success on Windows
  - Fixed a bug where using MACHINE_RESOURCE_NAMES could crash the startd
  - Improved condor c-gahp to prioritize commands over file transfers
  - Fixed a rare crash in the schedd when running many local universe jobs
  - With GSI, avoid unnecessary reverse DNS lookup when HOST_ALIAS is set
  - Fix a bug that could cause grid universe jobs to fail upon proxy refresh


author: Tim Theisen <tim@cs.wisc.edu> - 8.9.9-1
changelog:
  - The RPM packages requires globus, munge, scitokens, and voms from EPEL
  - Improved cgroup memory policy settings that set both hard and soft limit
  - Cgroup memory usage reporting no longer includes the kernel buffer cache
  - Numerous Python binding improvements, see version history
  - Can create a manifest of files on the execute node at job start and finish
  - Added provisioner nodes to DAGMan, allowing users to provision resources
  - DAGMan can now produce .dot graphs without running the workflow


author: Tim Theisen <tim@cs.wisc.edu> - 8.8.12-1
changelog:
  - Added a family of version comparison functions to ClassAds
  - Increased default Globus proxy key length to meet current NIST guidance


author: Tim Theisen <tim@cs.wisc.edu> - 8.9.10-1
changelog:
  - Fix bug where negotiator stopped making matches when group quotas are used
  - Support OAuth, SciTokens, and Kerberos credentials in local universe jobs
  - The Python schedd.submit method now takes a Submit object
  - DAGMan can now optionally run a script when a job goes on hold
  - DAGMan now provides a method for inline jobs to share submit descriptions
  - Can now add arbitrary tags to condor annex instances
  - Runs the "singularity test" before running the a singularity job


author: Tim Theisen <tim@cs.wisc.edu> - 8.9.11-1
changelog:
  - This release of HTCondor fixes security-related bugs described at
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2021-0001.html
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2021-0002.html


author: Tim Theisen <tim@cs.wisc.edu> - 8.8.13-1
changelog:
  - condor_ssh_to_job now maps CR and NL to work with editors like nano
  - Improved the performance of data transfer in condor_ssh_to_job
  - HA replication now accepts SHA-2 checksums to prepare for MD5 removal
  - Submission to NorduGrid ARC CE works with newer ARC CE versions
  - Fixed condor_annex crashes on platforms with newer compilers
  - Fixed "use feature: GPUsMonitor" to locate the monitor binary on Windows
  - Fixed a bug that prevented using the '@' character in an event log path


author: Tim Theisen <tim@cs.wisc.edu> - 8.9.13-1
changelog:
  - Host based security is no longer the default security model
  - Hardware accelerated integrity and AES encryption used by default
  - Normally, AES encryption is used for all communication and file transfers
  - Fallback to Triple-DES or Blowfish when interoperating with older versions
  - Simplified and automated new HTCondor installations
  - HTCondor now detects instances of multi-instance GPUs
  - Fixed memory leaks (collector updates in 8.9 could leak a few MB per day)
  - Many other enhancements and bug fixes, see version history for details


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.0-1
changelog:
  - Absent any configuration, HTCondor denies authorization to all users
  - AES encryption is used for all communication and file transfers by default
  - New IDTOKEN authentication method enables fine-grained authorization
  - IDTOKEN authentication method is designed to replace GSI
  - Improved support for GPUs, including machines with multiple GPUs
  - New condor_watch_q tool that efficiently provides live job status updates
  - Many improvements to the Python bindings
  - New Python bindings for DAGMan and chirp
  - Improved file transfer plugins supporting uploads and authentication
  - File transfer times are now recorded in the job log
  - Added support for jobs that need to acquire and use OAUTH tokens
  - Many memory footprint and performance improvements in DAGMan
  - Submitter ceilings can limit the number of jobs per user in a pool


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.1-1
changelog:
  - Fix problem where X.509 proxy refresh kills job when using AES encryption
  - Fix problem when jobs require a different machine after a failure
  - Fix problem where a job matched a machine it can't use, delaying job start
  - Fix exit code and retry checking when a job exits because of a signal
  - Fix a memory leak in the job router when a job is removed via job policy
  - Fixed the back-end support for the 'bosco_cluster --add' command
  - An updated Windows installer that supports IDTOKEN authentication


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.3-1
changelog:
  - Fixes for security issues
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2021-0003
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2021-0004


author: Tim Theisen <tim@cs.wisc.edu> - 8.8.15-1
changelog:
  - Fix for security issue
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2021-0003


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.5-1
changelog:
  - Other authentication methods are tried if mapping fails using SciTokens
  - Fix rare crashes from successful condor_submit, which caused DAGMan issues
  - Fix bug where ExitCode attribute would be suppressed when OnExitHold fired
  - condor_who now suppresses spurious warnings coming from netstat
  - The online manual now has detailed instructions for installing on MacOS
  - Fix bug where misconfigured MIG devices confused condor_gpu_discovery
  - The transfer_checkpoint_file list may now include input files


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.6-1
changelog:
  - CUDA_VISIBLE_DEVICES can now contain GPU-<uuid> formatted values
  - Fixed a bug that caused jobs to fail when using newer Singularity versions
  - Fixed a bug in the Windows MSI installer for the latest Windows 10 version
  - Fixed bugs relating to the transfer of standard out and error logs
  - MacOS 11.x now reports as 10.16.x (which is better than reporting x.0)


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.7-1
changelog:
  - Fix bug where condor_gpu_discovery could crash with older CUDA libraries
  - Fix bug where condor_watch_q would fail on machines with older kernels
  - condor_watch_q no longer has a limit on the number of job event log files
  - Fix bug where a startd could crash claiming a slot with p-slot preemption
  - Fix bug where a job start would not be recorded when a shadow reconnects


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.8-1
changelog:
  - Fix bug where huge values of ImageSize and others would end up negative
  - Fix bug in how MAX_JOBS_PER_OWNER applied to late materialization jobs
  - Fix bug where the schedd could choose a slot with insufficient disk space
  - Fix crash in ClassAd substr() function when the offset is out of range
  - Fix bug in Kerberos code that can crash on macOS and could leak memory
  - Fix bug where a job is ignored for 20 minutes if the startd claim fails


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.9-1
changelog:
  - Added Debian 11 (bullseye) as a supported platform
  - Since CentOS 8 has reached end of life, we build and test on Rocky Linux 8
  - The OAUTH credmon is now packaged for Enterprise Linux 8


author: Tim Theisen <tim@cs.wisc.edu> - 8.8.16-1
changelog:
  - Fix for security issue
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2022-0003


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.10-1
changelog:
  - Fixes for security issues
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2022-0001
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2022-0002
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2022-0003


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.11-1
changelog:
  - The Job Router can now create an IDTOKEN for use by the job
  - Fix bug where a self-checkpointing job may erroneously be held
  - Fix bug where the Job Router could erroneously substitute a default value
  - Fix bug where a file transfer error may identify the wrong file
  - Fix bug where condor_ssh_to_job may fail to connect


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.12-1
changelog:
  - Fix bug in parallel universe that could cause the schedd to crash
  - Fix rare crash where a daemon tries to use a discarded security session


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.13-1
changelog:
  - Schedd and startd cron jobs can now log output upon non-zero exit
  - condor_config_val now produces correct syntax for multi-line values
  - The condor_run tool now reports submit errors and warnings to the terminal
  - Fix issue where Kerberos authentication would fail within DAGMan
  - Fix HTCondor startup failure with certain complex network configurations


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.14-1
changelog:
  - SciToken mapping failures are now recorded in the daemon logs
  - Fix bug that stopped file transfers when output and error are the same
  - Ensure that the Python bindings version matches the installed HTCondor
  - $(OPSYSANDVER) now expand properly in job transforms
  - Fix bug where context managed Python htcondor.SecMan sessions would crash
  - Fix bug where remote CPU times would rarely be set to zero


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.16-1
changelog:
  - Singularity now mounts /tmp and /var/tmp under the scratch directory
  - Fix bug where Singularity jobs go on hold at the first checkpoint
  - Fix bug where gridmanger deletes the X.509 proxy file instead of the copy
  - Fix file descriptor leak when using SciTokens for authentication


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.17-1
changelog:
  - Fix file descriptor leak when schedd fails to launch scheduler jobs
  - Fix failure to forward batch grid universe job's refreshed X.509 proxy
  - Fix DAGMan failure when the DONE keyword appeared in the JOB line
  - Fix HTCondor's handling of extremely large UIDs on Linux
  - Fix bug where OAUTH tokens lose their scope and audience upon refresh
  - Support for Apptainer in addition to Singularity


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.2-1
changelog:
  - HTCondor can be set up to use only FIPS 140-2 approved security functions
  - If the Singularity test fails, the job goes idle rather than getting held
  - Can divide GPU memory, when making multiple GPU entries for a single GPU
  - Startd and Schedd cron job maximum line length increased to 64k bytes
  - Added first class submit keywords for SciTokens
  - Fixed MUNGE authentication
  - Fixed Windows installer to work when the install location isn't C:\Condor


author: Tim Theisen <tim@cs.wisc.edu> - 8.8.14-1
changelog:
  - Fix for security issue
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2021-0003.html


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.3-1
changelog:
  - Fixes for security issues
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2021-0003.html
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2021-0004.html


author: Tim Theisen <tim@cs.wisc.edu> - 8.8.14-1
changelog:
  - Fix for security issue
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2021-0003/


author: Tim Theisen <tim@cs.wisc.edu> - 9.0.3-1
changelog:
  - Fixes for security issues
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2021-0003/
  - https://research.cs.wisc.edu/htcondor/security/vulnerabilities/HTCONDOR-2021-0004/


