CSVLexer Class Reference

Tokenizer and record accumulator for CSV parsing. More...

#include <CSVLexer.hpp>

Inheritance diagram for CSVLexer:

Collaboration diagram for CSVLexer:

Public Member Functions
	CSVLexer (std::istream *in)
	Constructs a lexer bound to the given input stream.
virtual	~CSVLexer ()=default
	Virtual destructor; no additional cleanup beyond the base class.
auto	yylex (std::string *const yyval) -> int
	Scans the next token from the input stream.
auto	newline () -> void
	Begins a new record by appending an empty row to the result.
auto	append (const std::string &field) -> void
	Appends a field value to the current (last) record.
auto	data () const &-> const std::vector< std::vector< std::string > > &
	Returns a copy of all records accumulated so far.
auto	data () &&-> std::vector< std::vector< std::string > >
	Moves all records accumulated so far.
auto	reset () -> void
	Clears all accumulated records.

Detailed Description

Tokenizer and record accumulator for CSV parsing.

CSVLexer extends the Flex-generated yyFlexLexer base with:

• a typed yylex(std::string*) overload that returns the token kind and writes the matched text (for TEXTDATA tokens) into the caller-supplied string, matching the define api.value.type and YY_DECL signature configured in csv.l;
• record/field accumulation methods (newline(), append) that the Bison grammar actions in csv.y call as each field and record boundary is recognised;
• accessors (data(), reset()) for retrieving and clearing the accumulated result.

The lexer recognises five token kinds, defined in csv.l: CR (\r), LF (\n), COMMA (,), DQUOTE ("), and TEXTDATA (one or more printable ASCII characters excluding comma and double-quote). Token classification is the lexer's only responsibility; quote-doubling unescape logic and record/field structure are handled by the grammar actions in csv.y, which call back into this class via append() and newline().

Instances are short-lived : CSV::parse_helper() constructs one CSVLexer per parse, binds it to CSVParser, and discards it after extracting the result via data().

See also

pacemaker::io::CSV

Note

The generated CSVParser class (declared in the Bison-generated bison-csv.hpp, built from csv.y) is the sole caller of yylex(), append(), and newline().

Definition at line 39 of file CSVLexer.hpp.

Constructor & Destructor Documentation

CSVLexer()

CSVLexer ( std::istream * in )

inline

Constructs a lexer bound to the given input stream.

Forwards in to the yyFlexLexer base constructor, which uses it as the source of characters for tokenization.

Parameters

in	Pointer to the input stream to read from. Must outlive the lexer; ownership is not transferred.

Definition at line 50 of file CSVLexer.hpp.

~CSVLexer()

virtual ~CSVLexer ( )

virtualdefault

Virtual destructor; no additional cleanup beyond the base class.

Member Function Documentation

yylex()

auto yylex

(

std::string *const

yyval

)

-> int

Scans the next token from the input stream.

Implemented in the Flex-generated source produced from csv.l. For a matched TEXTDATA token, the matched text is copied into *yyval; for all other token kinds (CR, LF, COMMA, DQUOTE) *yyval is left unmodified, since the token kind alone is sufficient information for the grammar.

This signature matches the YY_DECL macro defined in csv.l and is invoked by the generated CSVParser as its yylex callback (see code { #define yylex lexer.yylex } in csv.y).

Parameters

yyval

Output parameter receiving the matched text for TEXTDATA tokens. Must not be null.

Returns

The recognised token kind, as a pacemaker::io::CSVParser::token enumerator value, or 0 at end of input.

newline()

auto newline ( ) -> void

inline

Begins a new record by appending an empty row to the result.

Called by the grammar action for the records rule in csv.y each time a complete record has been recognised, including after the first record. As a side effect of append()'s lazy-initialisation check, the very first record does not required an explicit newline() call before its first field is appended.

Definition at line 82 of file CSVLexer.hpp.

append()

auto append ( const std::string & field ) -> void

inline

Appends a field value to the current (last) record.

Called by the grammar action for the record rule in csv.y each time a field non-terminal is reduced.

Parameters

field

The unescaped field value, as computed by the grammar's escaped or nonescaped rule.

Definition at line 91 of file CSVLexer.hpp.

data() [1/2]

auto data ( ) const & -> const std::vector<std::vector<std::string>> &

inline

Returns a copy of all records accumulated so far.

Returns

A vector of records, each a vector of field strings, in the order they were parsed.

Definition at line 98 of file CSVLexer.hpp.

data() [2/2]

auto data ( ) && -> std::vector<std::vector<std::string>>

inline

Moves all records accumulated so far.

Returns

A vector of records, each a vector of field strings, in the order they were parsed.

Definition at line 105 of file CSVLexer.hpp.

reset()

auto reset ( ) -> void

inline

Clears all accumulated records.

After calling reset(), data() returns an empty vector until new records are accumulated via append() / newline(). Allows a single CSVLexer instance to be reused for multiple parses, although the current CSV::parse_helper() implementation constructs a fresh lexer per call instead of reusing one.

Definition at line 114 of file CSVLexer.hpp.

---

The documentation for this class was generated from the following file:

• src/pacemaker/io/CSVLexer.hpp