Ideas for furture improvements.  Not all are necessarily good or
feasible.

1) Done.

2) In dmk2cw, what if the DMK track buffer is a little too long to fit
   on the disk?  This is especially an issue now that cw2dmk uses a
   slightly longer buffer for reading and (in -h0 mode) might not put
   the original gap3 at the end of the buffer.  It's impossible to
   detect this error on MK1 or MK3, where the memory pointer can't be
   read back, but we can (and should) check for it on MK4.

3) In -h0 mode, we'd really like to find gap3 and start right after
   that, not with the first sector we see.  That would duplicate the
   original registration better, and also alleviate the worries in
   (2).  This is most easily and reliably done using a 2-pass
   algorithm: in pass 1, find the largest gap and deem it gap3; in
   pass 2, decode starting after gap3.

4) Copy the raw samples out of the CW into a buffer, then parse the
   buffer.  This allows us to do multi-pass algorithms in the future.
   (This is not strictly needed; calling catweasel_reset_pointer would
   already work.  However, it would be better to change the interface
   to work this way anyway, as it fits better with using a real driver
   in the future instead of touching the hardware directly in
   catweasl.c.)

4a) Support reading/writing raw samples to a file for later decoding.
   This will help with regression testing and debugging future tweaks
   to the decoder.

5) Done.

6) Done.  (Approximately.  We now print sector numbers in -v3.  This
   item had suggested doing it in -v2, but I prefer -v3.)

7) Done.

8) Check the timing under MS-DOS.  Stepping rate seems to be slow, and
   David Sutherland was having it miss steps (?) when restoring to 0.
   I think some drives can do that if you step at too wrong a speed.

9) The drive detection issue DS had could be something else too.  What
   if track0 output from his drive is invalid (perhaps always saying
   it *is* track 0) until the drive spins up and is ready?  Then we
   might get confused if we look at it too soon.  Just a wild guess.

10) Done.

11) Done.

12) Change cw2dmk and dmk2cw to work with a real device driver.

    * On Linux, use the cw driver from Karsten Scheibler.  May need to
      add MK1 and possibly also MK3 support to this driver.

    * On Windows, the driver from Tobias at Individual Computers
      should be adequate.  He has done a port that I can adapt.
      However, I need to check if anything is broken there.  There is
      a kludge that rejects tracks with 16 or 17 good sectors but no
      errors, and I'm wondering what necessitated that.  Also, do both
      -h0 and -h1 modes work?  Does this driver have MK1 support?  MK3?

    * Tim Lindner did a Mac port, and Tobias's sources seem to be
      based on that, but I think the Mac-specific code is missing.
      Can try to roll that in too, but will need help (perhaps from
      Tim L.) to test it.

13) dmk2cw might benefit from a still more sophisticated write-precomp
    algorithm, if I can find one described anywhere.  Right now only
    short (10) intervals are compensated, and the amount of
    compensation is the same regardless of whether the adjacent
    interval is medium (100) or long (1000).

14) Done.

15) Done.

16) Done.

17) Done.

18) Read additional command line options out of a file.  Say, .cw2dmk
    or cw2dmk.ini.  Ira and D.S. keep bugging me for this.

19) When the FM/MFM detector starts out guessing MFM at the beginning
    of a track but it's really FM there, it can't help reading much of
    the initial gap incorrectly (that is, in MFM) before it figures
    out the problem and switches.  That results in the track being
    recorded in the DMK file as if the gap started with a long run of
    00 00 00...  instead of FF FF FF....  (It could randomly be either
    depending on which bits are interpreted as clock and which as
    data, but the detector has a heuristic that says 00 is expected in
    MFM gaps rather than FF, so it actively shifts to 00.)  Having
    00's instead of FF's in the gap doesn't hurt the sector headers
    and user data any, but if some software is looking at gap
    contents, the initial gap will have the wrong thing in it.  This
    case gets triggered if track N is (or starts with) MFM and track
    N+1 is (or starts with) FM.  I don't have any ideas on fixing it;
    it's a dilemma even with the new decoder.

20) Done, but 20a-20c remain as fallout.

20a) Done.

20b) An FM 0xf9 DAM is ambiguous.  We have to consider it as plain FM
    (a WD1771 extension) by default, but as RX02-MFM if -e3 was
    specified or we've seen any 0xfd DAMs on this disk.  We should
    really detect the pathological case where we see an 0xf9 and
    interpret it as WD1771-FM, then later see a 0xfd and realize this
    is an RX02 disk.  This case will also have a CRC error in the 0xf9
    sector, so we'll still be OK if there is an 0xfd sector on the
    same track -- we'll decode the 0xf9 sector correctly on the first
    retry.  Fortunately -e3 is a fine workaround, and there is enough
    guidance in the man page that people should think to try it.

20c) Done.

21) For non-hole aligned reads, does it make sense to read exactly
    until the CW's memory is full (128K)?  Maybe not, as that's quite
    a variable amount of time.  How close are we to filling it on one
    or two revs of (say) HD data?  Answer: On 3.5" HD, the data rate
    is 500 kHz and there are 5 revs per second, so there can be up to
    about 100,000 transitions.  So about 1.3 revs fit in memory.  This
    means -h0 won't always work if sectors are too long, but they
    usually aren't.

22) Consider having detect_kind dynamically set the thresholds for a
    track by examining the histogram.  (This will be especially good if
    a track contains only FM and thus has only two peaks; we can
    automatically have only one threshold then.)  Jim Battle's "helios"
    program for decoding Sol Helios/PTDOS hard sectored disks (based
    in part on cw2dmk) does this.  I'm a bit worried it will go wrong
    sometimes, though.  Looking at the comments and stuff in Jim's
    adaptation of my code, he had problems with it finding too many
    peaks due (he thought) to variations in disk drive speed, so the
    peak detector probably needs more work to be reliable enough.

23) Learn some real signal processing algorithms instead of just
    making stuff up.  Unfortunately my web searches haven't yet turned
    up anything I can use.

    23a) One might think a software PLL would be better than fixed
    thresholds for detecting sample lengths.  I'm not sure that's
    true, though.  The idea of using the histogram of the whole track
    (see 22) seems intuitively like it should be better: The frequency
    is fairly constant, so using the whole track's information
    together should give a more accurate estimate than trying to
    dynamically track it with an on-line, sample-by-sample algorithm.
    Even if different sectors were written in different drives at
    slightly different speeds, a per-sector histogram may be better
    than a PLL.  Anyway, I didn't find a cookbook PLL algorithm I
    could use, and my attempts to kludge one up were a failure.

    23b) Surely there must be better algorithms for peak detection in
    the literature than the one I invented ad hoc for
    testhist/detect_kind (see 22).  It seems like correlating or
    convolving the histogram with what we expect it to look like ought
    to help...?

    23c) An algorithm for detecting wraparound, using the fact that at
    some point the sample stream starts to approximately match the
    stream seen at the beginning of the track, would be nice.

    23d) It might be useful to be able to average together several
    passes over the same track.  This has to be done carefully since
    if there are "weak bits" (even if unintentional), one pass might
    have extra samples inserted where the other has none.

24) Factor out common code in testhist.c and cw2dmk's detect_kind().

25) Should we drop the first sample on the MK4?  It looks sort of
    garbagey.  What about on MK1 and MK3?  (On those we're already
    dropping two samples because of the problem with not knowing
    whether we wrapped, so perhaps it's moot there.)

26) It would be interesting to remember places where we were unsure
    about thresholds (or other decisions), and go back and retry those
    decisions the other way if one fell inside a sector with a CRC
    error (or part of a track with some other error).  It seems like
    the "maximum likelihood" concept may fit in here somewhere.

27) On the MK4 it's once again possible to wait in hardware to start a
    read on the first rising index edge (although you then have to
    stop it in software or wait until the memory fills up).  Should
    use this to make reading more reliable: on an MK3, we can
    sometimes miss some data right after the index hole if the OS
    takes an interrupt or otherwise deschedules cw2dmk while it's
    spinning waiting for the hole to show up.  (It would also be good
    to have a way to avoid this problem on the MK3.)  Maybe this idea
    is obsolete if I'm going to use a real driver; have to see what
    the driver provides.

28) Done.

29) dmk2cw could use a way to vary the data rate slightly to adjust
    the track length.  Basically, a flag to alter the mfmshort value
    read from the kind descriptor would do it.  How can this be done
    most conveniently?  Exact track length doesn't matter for any
    TRS-80 disks I know of, but it does for some Amiga copy
    protections, and I'm not sure if it does for PC Everlock's
    wraparound sector.  So it's worth some thought.

30) cw2dmk could maybe use a flag to always cut off at index edge 3
    and log the exact track length found.  In DMK format, track length
    is global to the whole disk, unfortunately, but we could at least
    log it.  For Amiga and other formats where exact track length may
    be critical, we'll need at least the ability for the user to tweak
    it, and possible also algorithms to find the correct overlap
    automatically from the data, with finer grain than just "where the
    index edge is" (hard to do).

31) Done.

32) When retries are needed, should keep the "best" read so far and
    save that one if we never get an error-free read.  

33) Even better (but harder), we may occasionally be able to piece
    together a better track by taking sectors from different retries.

34) When the physical track ends, cw2dmk fills the rest of the DMK
    track with zeros.  Is this best?  It could be important because
    that data will be written out again by dmk2cw if the drive speed
    is slower (or data rate is faster) on writing.  Using 00 is
    inconsistent with the new fill feature in dmk2cw when the DMK
    track is too short.

35) The limitations of DMK format are a problem at times.  Partial list:

    * No way to record the actual length read from a physical track.
      This makes it impossible to represent formats where that matters
      or to even try to get dmk2cw to write them back out exactly.

    * The IDAM pointers are a very limited way to indicate where the
      format has missing clocks.  In dmk2cw, we already just guess
      where IAMs are.

    * Obviously unsuitable for formats that stray far from FM/MFM.

    I'm not sure what's best to do about this, if anything.  In
    general, I would rather use a good file format that someone else
    is using than invent a new one or make incompatible changes to DMK.
