Machine Learning Laboratory

Change Log

1. Fixed tree save/restore. Format of .iti files has changed.
2. Added function to delete an instance from the current tree.

1. Fixed dmti - it was not installing the intended test at a decision node.

1. Rewrote cpu_used() to use library function times(), which is in standard library. That function returns times in clock ticks, so there is extra code to convert those to seconds, based on number of ticks per second, obtained via call to sysconf(). The getrusage(), which was nonstandard, is now gone.
2. The cpu_used() function counts both user and sys time - formerly it counted just user time. Cpu results with this version should not be compared to cpu results from earlier versions.

1. Fixed problem in -M option. It may have been computing the number of possible cutpoints at a node incorrectly for the purpose of computing minimum description length.

1. Input scanner trims leading and trailing blanks from variable values and class names.

1. Moved two #defines from iti.h to iti_pst.h
2. Changed several %d formatters in plot.c to %3.2lf

1. sprint argument for error message missing in dmti.c

1. Transpose_tree fixed.

1. Parentheses in variables names or symbolic values are removed automatically when loaded.
2. Counting error in transpose_tree fixed.

1. Removed Goodman-Kruskal indirect metric.
2. Removed gain-ratio indirect metric.
3. Made Kolmogorov-Smirnoff distance the only indirect metric. Code reworked.
4. Tree save-to-file and restore-from-file rewritten. Tree now saved in an ascii form. Still not for human consumption, but portable.
5. Reorganized the .h files. There is now a config.h file with a few constants in it. No need to look at anything else.
6. All known bugs eliminated from dmti. Appears bug free.
7. Data structure for values of instance changed to be a dynamically allocated array.
8. Variable names stored globally just once. No longer anywhere in tree.
9. Source files reorganized. The iti.c file is now an application that makes use of the library routines. An API will be documented soon.
10. Leave-one-out and k-fold cross validation code removed.
11. Data structure for variables changed to be a dynamically allocated array. (It used to be an AVL tree) Much code changed because of this. There is an improvement in both time and space.
12. For PST, tree arcs now have `t' and `f' labels.
13. For PST, default font size is now 6 (was 8).
14. Switched to Web-based documentation.

1. Added Goodman-Kruskal indirect metric.
2. Added minimum number of instances of second majority class that are needed to consider leaf as impure. Default is 1.
3. When loading instances, the value "NAN" is equated with "?", i.e. missing.

1. Instances with missing values are now handled differently. It is now assumed that an instance with a missing value fails the test at a node. This means that the FALSE branch is taken. This is so for both training and testing. Many coding changes.
2. Majority class now breaks ties in favor of classname that is lexocographically less.
3. Cutpoints that would break pure intervals (for numeric variables) no longer evaluated. This may also change tree that will be built as it changes the average gain.
4. xval-prep no longer prints to stderr.
5. class_counts now of type int.
6. Number of nodes is now often broken into number of leaves and number of decision nodes. The 'N' option becomes the 'L' option.

1. Fixed bug in transpose_tree.

1. Converted to ANSII C.
2. A tie (according to attribute selection metric) between two symbolic values of a variable is now broken in favor of the lexicographically lesser of the two. (Formerly, it was the case that symbolic value with the lower address was preferred, which can cause some problems in systems that embed ITI.) Similarly, a tie between two variables is now broken in favor of the lexicographically lesser of the two.
3. Added ability to select attribute selection metric. Instead of always using gain-ratio, one can select kolmogorov_smirnov. Default is gain-ratio. See iti.doc for more info. Some other metrics may be added in the future.
4. Renamed variety of procedures/functions related to gain_ratio. Distinguishes general parts of metric from specific metric used.
5. Brought xval.sh up to date.

1. Added k-fold cross-validation.
2. Replaced each call to nint() with call to rint() in information.c.

1. Removed an extraneous line of code.
2. Fixed error-correction mode.
3. Fixed plot.c and added a plot.doc.

1. Fixed printing of tree, was omitting symbolic attribute value.
2. Changed `optimize' namings to `dmti', i.e. direct metric tree induction. This conforms to how system is now described in TR-95-18.
3. Added restore_tree() and save_tree().
4. Added leave-one-out cross validation.
5. Reworked specification of arguments on command line.
6. Added new tech report (TR 95-18) describing ITI and DMTI.

1. Fixed documentation regarding command line arguments.
2. Fixed scanner to allow instances without a trailing '.'.
3. Reworked optimize.c to create separate list of tests at a node before trying each one. Necessary because root node of any tree/subtree may change due to tree restructuring. Can't pick the tests out of a node because node can change.

1. Reworked how transposition is done for tests with numeric variables. Got rid of `slewing'. Just do a regular transposition without any special handling.