boost-react-native-bundle

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head><title>Trees</title> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <link rel="stylesheet" href="theme/style.css" type="text/css"></head> <body> <table width="100%" border="0" background="theme/bkd2.gif" cellspacing="2"> <tbody><tr> <td width="10"> </td> <td width="85%"> Trees </td> <td width="112"><a href="http://spirit.sf.net"><img src="theme/spirit.gif" width="112" height="48" align="right" border="0"></a></td> </tr> </tbody></table> <table border="0"> <tbody><tr> <td width="10"> </td> <td width="30"><a href="../index.html"><img src="theme/u_arr.gif" border="0"></a></td> <td width="30"><a href="symbols.html"><img src="theme/l_arr.gif" border="0"></a></td> <td width="30"><a href="multi_pass.html"><img src="theme/r_arr.gif" border="0"></a></td> </tr> </tbody></table> <h2>Why use parse trees</h2> Parse trees are an in-memory representation of the input with a structure that conforms to the grammar. The advantages of using parse trees instead of semantic actions: <ul> <li>You can make multiple passes over the data without having to re-parse the input.</li> <li>You can perform transformations on the tree.</li> <li>You can evaluate things in any order you want, whereas with attribute schemes you have to process in a begin to end fashion.</li> <li>You do not have to worry about backtracking and action side effects that may occur with an ambiguous grammar.</li> </ul> Example Now that you think you may want to use trees, I'll give an example of how to use them and you can see how easy they are to use. So, following with tradition (and the rest of the documentation) we'll do a calculator. Here's the grammar: <pre> <code>integer = token_node_d[ (!ch_p('-') >> +digit_p) ] ; factor = integer | '(' >> expression >> ')' | ('-' >> factor) ; term = factor >> *( ('*' >> factor) | ('/' >> factor) ) ; expression = term >> *( ('+' >> term) | ('-' >> term) ) ;</code></pre> Now, you'll notice the only thing different in this grammar is the <tt>token_node_d</tt> directive. This causes the match of the integer rule to end up in one node. Without <tt>token_node_d</tt>, each character would get it's own node. Further note that <tt>token_node_d</tt> is an implicit lexeme (that means no <tt>lexeme_d</tt> is needed to switch to character level parsing). As you'll soon see, it's easier to convert the input into an int when all the characters are in one node. Here is how the parse is done to create a tree: <pre> <code>tree_parse_info<> info = pt_parse(first, expression);</code></pre> <tt>pt_parse()</tt> is similar to <tt>parse()</tt>. There are four overloads: two for pairs of first and last iterators and two for character strings. Two of the functions take a skipper parser and the other two do not. The <tt>tree_parse_info</tt> struct contains the same information as a <tt>parse_info</tt> struct as well as one extra data member called trees. When the parse finishes, trees will contain the parse tree. Here is how you can use the tree to evaluate the input: <pre> <code>if (info.full) { cout << "parsing succeeded\n"; cout << "result = " << evaluate(info) << "\n\n"; }</code></pre> Now you ask, where did <tt>evaluate()</tt> come from? Is it part of spirit? Unfortunately, no, <tt>evaluate()</tt> is only a part of the sample. Here it is: <pre> <code>long evaluate(const tree_parse_info<>& info) { return eval_expression(info.trees.begin()); }</code></pre> So here you see evaluate() simply calls <tt>eval_expression()</tt> passing the begin() iterator of info.trees. Now here's the rest of the example: <pre> <code>// Here's some typedefs to simplify things typedef char const* iterator_t; typedef tree_match<iterator_t> parse_tree_match_t; typedef parse_tree_match_t::const_tree_iterator iter_t; // Here's the function prototypes that we'll use. One function for each // grammar rule. long evaluate(const tree_parse_info<>& info); long eval_expression(iter_t const& i); long eval_term(iter_t const& i); long eval_factor(iter_t const& i); long eval_integer(iter_t const& i); // i should be pointing to a node created by the expression rule long eval_expression(iter_t const& i) { // first child points to a term, so call eval_term on it iter_t chi = i->children.begin(); long lhs = eval_term(chi); for (++chi; chi != i->children.end(); ++chi) { // next node points to the operator. The text of the operator is // stored in value (a vector<char>) char op = *(chi->value.begin()); ++chi; long rhs = eval_term(chi); if (op == '+') lhs += rhs; else if (op == '-') lhs -= rhs; else assert(0); } return lhs; } long eval_term(iter_t const& i) { // ... see <a href="../example/fundamental/parse_tree_calc1.cpp">parse_tree_calc1.cpp</a> for complete example // (it's rather similar to eval_expression() ) ... } long eval_factor(iter_t const& i) { // ... again, see <a href="../example/fundamental/parse_tree_calc1.cpp">parse_tree_calc1.cpp</a> if you want all the details ... } long eval_integer(iter_t const& i) { // use the range constructor for a string string integer(i->value.begin(), i->value.end()); // convert the string to an integer return strtol(integer.c_str(), 0, 10); } </code></pre> <img height="16" width="15" src="theme/lens.gif"> The full source code can be <a href="../example/fundamental/parse_tree_calc1.cpp">viewed here</a>. This is part of the Spirit distribution. So, you like what you see, but maybe you think that the parse tree is too hard to process? With a few more directives you can generate an abstract syntax tree (ast) and cut the amount of evaluation code by at least 50%. So without any delay, here's the ast calculator grammar: <pre> <code>integer = leaf_node_d[ (!ch_p('-') >> +digit_p) ] ; factor = integer | inner_node_d[ch_p('(') >> expression >> ch_p(')')] | (root_node_d[ch_p('-')] >> factor) ; term = factor >> *( (root_node_d[ch_p('*')] >> factor) | (root_node_d[ch_p('/')] >> factor) ) ; expression = term >> *( (root_node_d[ch_p('+')] >> term) | (root_node_d[ch_p('-')] >> term) ) ;</code></pre> The differences from the parse tree grammar are hi-lighted in bold-red. The <tt>inner_node_d</tt> directive causes the first and last nodes generated by the enclosed parser to be discarded, since we don't really care about the parentheses when evaluating the expression. The <tt>root_node_d</tt> directive is the key to ast generation. A node that is generated by the parser inside of <tt>root_node_d</tt> is marked as a root node. When a root node is created, it becomes a root or parent node of the other nodes generated by the same rule. To start the parse and generate the ast, you must use the ast_parse functions, which are similar to the pt_parse functions. <pre> <code>tree_parse_info<> info = ast_parse(first, expression);</code></pre> Here is the eval_expression function (note that to process the ast we only need one function instead of four): <pre> <code>long eval_expression(iter_t const& i) { if (i->value.id() == parser_id(&integer)) { // convert string to integer string integer(i->value.begin(), i->value.end()); return strtol(integer.c_str(), 0, 10); } else if (i->value.id() == parser_id(&factor)) { // factor can only be unary minus return - eval_expression(i->children.begin()); } else if (i->value.id() == parser_id(&term)) { if (*i->value.begin() == '*') { return eval_expression(i->children.begin()) * eval_expression(i->children.begin()+1); } else if (*i->value.begin() == '/') { return eval_expression(i->children.begin()) / eval_expression(i->children.begin()+1); } } else if (i->value.id() == parser_id(&expression)) { if (*i->value.begin() == '+') { return eval_expression(i->children.begin()) + eval_expression(i->children.begin()+1); } else if (*i->value.begin() == '-') { return eval_expression(i->children.begin()) - eval_expression(i->children.begin()+1); } } return 0; } </code></pre> <img src="theme/lens.gif" width="15" height="16"> An entire working example is <a href="../example/fundamental/ast_calc.cpp">ast_calc.cpp</a>. Hopefully this example has been enough to whet your appetite for trees. For more nitty-gritty details, keep on reading the rest of this chapter. <a name="usage"></a> <h2>Usage</h2> <a name="pt_parse"></a> <h3>pt_parse</h3> To create a parse tree, you can call one of the five free functions: <pre> template <typename FactoryT, typename IteratorT, typename ParserT, typename SkipT> tree_parse_info<IteratorT, FactoryT> pt_parse( IteratorT const& first_, IteratorT const& last_, parser<ParserT> const& parser, SkipT const& skip_, FactoryT const & factory_ = FactoryT()); template <typename IteratorT, typename ParserT, typename SkipT> tree_parse_info<IteratorT> pt_parse( IteratorT const& first_, IteratorT const& last_, parser<ParserT> const& parser, SkipT const& skip_); template <typename IteratorT, typename ParserT> tree_parse_info<IteratorT> pt_parse( IteratorT const& first_, IteratorT const& last_, parser<ParserT> const& parser); template <typename CharT, typename ParserT, typename SkipT> tree_parse_info<CharT const*> pt_parse( CharT const* str, parser<ParserT> const& parser, SkipT const& skip); template <typename CharT, typename ParserT> tree_parse_info<CharT const*> pt_parse( CharT const* str, parser<ParserT> const& parser); </pre> <a name="ast_parse"></a> <h3>ast_parse</h3> To create an abstract syntax tree (ast for short) you call one of the five free functions: <pre> template <typename FactoryT, typename IteratorT, typename ParserT, typename SkipT> tree_parse_info<IteratorT, FactoryT> ast_parse( IteratorT const& first_, IteratorT const& last_, parser<ParserT> const& parser, SkipT const& skip_, FactoryT const & factory_ = FactoryT()); template <typename IteratorT, typename ParserT, typename SkipT> tree_parse_info<IteratorT> ast_parse( IteratorT const& first_, IteratorT const& last_, parser<ParserT> const& parser, SkipT const& skip_); template <typename IteratorT, typename ParserT> tree_parse_info<IteratorT> ast_parse( IteratorT const& first_, IteratorT const& last, parser<ParserT> const& parser); template <typename CharT, typename ParserT, typename SkipT> tree_parse_info<CharT const*> ast_parse( CharT const* str, parser<ParserT> const& parser, SkipT const& skip); template <typename CharT, typename ParserT> tree_parse_info<CharT const*> ast_parse( CharT const* str, parser<ParserT> const& parser); </pre> <a name="tree_parse_info"></a> <h3>tree_parse_info</h3> The <tt>tree_parse_info</tt> struct returned from pt_parse and ast_parse contains information about the parse: <pre> <code>template <typename IteratorT = char const*> struct tree_parse_info { IteratorT stop; bool match; bool full; std::size_t length; typename tree_match<IteratorT>::container_t trees; }; </code></pre> <table width="90%" border="0" align="center"> <tbody><tr> <td class="table_title" colspan="10"> tree_parse_info </td> </tr> <tr> </tr><tr> <td class="table_cells">stop</td> <td class="table_cells">points to the final parse position (i.e. parsing processed the input up to this point).</td> </tr> <tr><td class="table_cells">match</td> <td class="table_cells">true if parsing is successful. This may be full (the parser consumed all the input), or partial (the parser consumed only a portion of the input.)</td> </tr> <tr><td class="table_cells">full</td> <td class="table_cells">true when we have a full match (when the parser consumed all the input).</td> </tr> <tr><td class="table_cells">length</td> <td class="table_cells">The number of characters consumed by the parser. This is valid only if we have a successful match (either partial or full).</td> </tr> <tr><td class="table_cells">trees</td> <td class="table_cells">Contains the root node(s) of the tree.</td> </tr> </tbody></table> <a name="tree_match"></a> <h3>tree_match</h3> When Spirit is generating a tree, the parser's parse() member function will return a tree_match object, instead of a match object. tree_match has three template parameters. The first is the Iterator type which defaults to <tt>char const*</tt>. The second is the node factory, which defaults to <a href="#node_val_data_factory">node_val_data_factory</a>. The third is the attribute type stored in the match. A tree_match has a member variable which is a container (a <tt>std::vector</tt>) of <a href="#tree_node">tree_node</a> objects named trees. For efficiency reasons, when a tree_match is copied, the trees are not copied, they are moved to the new object, and the source object is left with an empty tree container. tree_match supports the same interface as the match class: it has an operator bool() so you can test it for a sucessful match: if (matched), and you can query the match length via the length() function. The class has this interface: <pre> <code>template <typename IteratorT = char const*, typename NodeFactoryT = node_val_data_factory<> > struct tree_match { typedef typename NodeFactoryT::template factory<IteratorT> node_factory_t; typedef typename node_factory_t::node_t parse_node_t; typedef tree_node<parse_node_t> node_t; typedef typename node_t::children_t