Hidden Markov modeling for single channel kinetics with filtering and correlated noise

Hidden Markov modeling (HMM) can be applied to extract single channel kinet ics at signal-to-noise ratios that are too low for conventional analysis. T here are two general HMM approaches: traditional Baum's reestimation and di rect optimization. The optimization approach has the advantage that it opti mizes the rate constants directly. This allows setting constraints on the r ate constants, fitting multiple data sets across different experimental con ditions, and handling nonstationary channels where the starting probability of the channel depends on the unknown kinetics. We present here an extensi on of this approach that addresses the additional issues of low-pass filter ing and correlated noise. The filtering is modeled using a finite impulse r esponse (FIR) filter applied to the underlying signal, and the noise correl ation is accounted for using an autoregressive (AR) process, In addition to correlated background noise, the algorithm allows for excess open channel noise that can be white or correlated. To maximize the efficiency of the al gorithm, we derive the analytical derivatives of the likelihood function wi th respect to all unknown model parameters. The search of the likelihood sp ace is performed using a variable metric method. Extension of the algorithm to data containing multiple channels is described. Examples are presented that demonstrate the applicability and effectiveness of the algorithm. Prac tical issues such as the selection of appropriate noise AR orders are also discussed through examples.