Our JSON parser is a three stage parser. The first stage tokenizes the stream
into a set of lexical tokens. Since the lexical grammar is regular, we can
use a finite state machine to model it. The state machine will emit tokens
as they are identified.
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
