ACE_Tokenizer Class Reference

Tokenizer. More...

#include <SString.h>

Collaboration diagram for ACE_Tokenizer:

[legend]List of all members.

Public Types
enum	{ MAX_DELIMITERS = 16, MAX_PRESERVES = 16 }
Public Member Functions
	ACE_Tokenizer (ACE_TCHAR *buffer)
int	delimiter (ACE_TCHAR d)
int	delimiter_replace (ACE_TCHAR d, ACE_TCHAR replacement)
int	preserve_designators (ACE_TCHAR start, ACE_TCHAR stop, int strip=1)
ACE_TCHAR *	next (void)
	Returns the next token.
Protected Member Functions
int	is_delimiter (ACE_TCHAR d, int &replace, ACE_TCHAR &r)
int	is_preserve_designator (ACE_TCHAR start, ACE_TCHAR &stop, int &strip)
Protected Attributes
ACE_TCHAR *	buffer_
int	index_
Preserve_Entry	preserves_ [MAX_PRESERVES]
	The application can specify MAX_PRESERVES preserve designators.
int	preserves_index_
	Pointer to the next free spot in preserves_.
Delimiter_Entry	delimiters_ [MAX_DELIMITERS]
	The tokenizer allows MAX_DELIMITERS number of delimiters.
int	delimiter_index_
	Pointer to the next free space in delimiters_.

---

Detailed Description

Tokenizer.

Tokenizes a buffer. Allows application to set delimiters and preserve designators. Does not allow special characters, yet (e.g., printf ("\"like a quoted string\"")).

Definition at line 271 of file SString.h.

---

Member Enumeration Documentation

anonymous enum

Enumeration values: MAX_DELIMITERS  MAX_PRESERVES  Definition at line 381 of file SString.h. { MAX_DELIMITERS=16, MAX_PRESERVES=16 };

---

Constructor & Destructor Documentation

ACE_Tokenizer::ACE_Tokenizer (  ACE_TCHAR *  buffer  )

buffer will be parsed. Notice that ACE_Tokenizer will modify buffer if you use delimiter_replace or preserve_designators to do character substitution. Note: You should NOT pass a constant string or string literal to this constructor, since ACE_Tokenizer will try to modify the string. See also: preserve_designators preserve_designators Definition at line 339 of file SString.cpp. References ACE_TCHAR. : buffer_ (buffer), index_ (0), preserves_index_ (0), delimiter_index_ (0) { }

---

Member Function Documentation

int ACE_Tokenizer::delimiter (  ACE_TCHAR  d  )

d is a delimiter. Returns: Returns 0 on success, -1 if there is no memory left. Example: char buf[30]; ACE_OS::strcpy(buf, "William/Joseph/Hagins"); ACE_Tokenizer tok (buf); tok.delimiter ('/'); for (char *p = tok.next (); p; p = tok.next ()) cout << p << endl; This will print out: William/Joseph/Hagins Joseph/Hagins Hagins Definition at line 348 of file SString.cpp. References ACE_TCHAR, ACE_Tokenizer::Delimiter_Entry::delimiter_, delimiter_index_, delimiters_, MAX_DELIMITERS, and ACE_Tokenizer::Delimiter_Entry::replace_. { if (delimiter_index_ == MAX_DELIMITERS) return -1; delimiters_[delimiter_index_].delimiter_ = d; delimiters_[delimiter_index_].replace_ = 0; delimiter_index_++; return 0; }

int ACE_Tokenizer::delimiter_replace (  ACE_TCHAR  d, ACE_TCHAR  replacement )

d is a delimiter and, when found, will be replaced by replacement. Returns: 0 on success, -1 if there is no memory left. Example: char buf[30]; ACE_OS::strcpy(buf, "William/Joseph/Hagins"); ACE_Tokenizer tok (buf); tok.delimiter_replace ('/', 0); for (char *p = tok.next (); p; p = tok.next ()) cout << p << endl; This will print out: William Joseph Hagins Definition at line 360 of file SString.cpp. References ACE_TCHAR, ACE_Tokenizer::Delimiter_Entry::delimiter_, delimiter_index_, delimiters_, MAX_DELIMITERS, ACE_Tokenizer::Delimiter_Entry::replace_, and ACE_Tokenizer::Delimiter_Entry::replacement_. Referenced by ACE_Process_Options::command_line_argv(), ACE_Configuration::expand_path(), and ACE_Configuration_Win32Registry::resolve_key(). { // Make it possible to replace delimiters on-the-fly, e.g., parse // string until certain token count and then copy rest of the // original string. for (int i = 0; i < delimiter_index_; i++) if (delimiters_[i].delimiter_ == d) { delimiters_[i].replacement_ = replacement; delimiters_[i].replace_ = 1; return 0; } if (delimiter_index_ >= MAX_DELIMITERS) return -1; delimiters_[delimiter_index_].delimiter_ = d; delimiters_[delimiter_index_].replacement_ = replacement; delimiters_[delimiter_index_].replace_ = 1; delimiter_index_++; return 0; }

int ACE_Tokenizer::is_delimiter (  ACE_TCHAR  d, int &  replace, ACE_TCHAR &  r )  [protected]

Returns 1 if is a delimiter, 0 otherwise. If should be replaced with r, is set to 1, otherwise 0. Definition at line 400 of file SString.cpp. References ACE_TCHAR, ACE_Tokenizer::Delimiter_Entry::delimiter_, delimiter_index_, delimiters_, ACE_Tokenizer::Delimiter_Entry::replace_, and ACE_Tokenizer::Delimiter_Entry::replacement_. Referenced by next(). { replace = 0; for (int x = 0; x < delimiter_index_; x++) if (delimiters_[x].delimiter_ == d) { if (delimiters_[x].replace_) { r = delimiters_[x].replacement_; replace = 1; } return 1; } return 0; }

int ACE_Tokenizer::is_preserve_designator (  ACE_TCHAR  start, ACE_TCHAR &  stop, int &  strip )  [protected]

If is a start preserve designator, returns 1 and sets to the stop designator. Returns 0 if is not a preserve designator. Definition at line 421 of file SString.cpp. References ACE_TCHAR, preserves_, preserves_index_, ACE_Tokenizer::Preserve_Entry::start_, and ACE_Tokenizer::Preserve_Entry::stop_. Referenced by next(). { for (int x = 0; x < preserves_index_; x++) if (preserves_[x].start_ == start) { stop = preserves_[x].stop_; strip = preserves_[x].strip_; return 1; } return 0; }

ACE_TCHAR * ACE_Tokenizer::next (  void   )

Returns the next token. Definition at line 437 of file SString.cpp. References ACE_TCHAR, is_delimiter(), and is_preserve_designator(). Referenced by ACE_Process_Options::command_line_argv(), ACE_Configuration::expand_path(), and ACE_Configuration_Win32Registry::resolve_key(). { // Check if the previous pass was the last one in the buffer. if (index_ == -1) { index_ = 0; return 0; } ACE_TCHAR replacement = 0; int replace; ACE_TCHAR *next_token; // Skip all leading delimiters. for (;;) { // Check for end of string. if (buffer_[index_] == '\0') { // If we hit EOS at the start, return 0. index_ = 0; return 0; } if (this->is_delimiter (buffer_[index_], replace, replacement)) index_++; else break; } // When we reach this point, buffer_[index_] is a non-delimiter and // not EOS - the start of our next_token. next_token = buffer_ + index_; // A preserved region is it's own token. ACE_TCHAR stop; int strip; if (this->is_preserve_designator (buffer_[index_], stop, strip)) { while (++index_) { if (buffer_[index_] == '\0') { index_ = -1; goto EXIT_LABEL; } if (buffer_[index_] == stop) break; } if (strip) { // Skip start preserve designator. next_token += 1; // Zap the stop preserve designator. buffer_[index_] = '\0'; // Increment to the next token. index_++; } goto EXIT_LABEL; } // Step through finding the next delimiter or EOS. for (;;) { // Advance pointer. index_++; // Check for delimiter. if (this->is_delimiter (buffer_[index_], replace, replacement)) { // Replace the delimiter. if (replace != 0) buffer_[index_] = replacement; // Move the pointer up and return. index_++; goto EXIT_LABEL; } // A preserve designator signifies the end of this token. if (this->is_preserve_designator (buffer_[index_], stop, strip)) goto EXIT_LABEL; // Check for end of string. if (buffer_[index_] == '\0') { index_ = -1; goto EXIT_LABEL; } } EXIT_LABEL: return next_token; }

int ACE_Tokenizer::preserve_designators (  ACE_TCHAR  start, ACE_TCHAR  stop, int  strip = 1 )

Extract string between a pair of designator characters. For instance, quotes, or '(' and ')'. start specifies the begin designator. stop specifies the end designator. strip If strip == 1, then the preserve designators will be stripped from the tokens returned by next. Returns: 0 on success, -1 if there is no memory left. Example with strip = 0: char buf[30]; ACE_OS::strcpy(buf, "William(Joseph)Hagins"); ACE_Tokenizer tok (buf); tok.preserve_designators ('(', ')', 0); for (char *p = tok.next (); p; p = tok.next ()) cout << p << endl; This will print out: William(Joseph)Hagins (Joseph)Hagins )Hagins Example with strip = 1: char buf[30]; ACE_OS::strcpy(buf, "William(Joseph)Hagins"); ACE_Tokenizer tok (buf); tok.preserve_designators ('(', ')', 1); for (char *p = tok.next (); p; p = tok.next ()) cout << p << endl; This will print out: William Joseph Hagins Definition at line 385 of file SString.cpp. References ACE_TCHAR, MAX_PRESERVES, preserves_, preserves_index_, ACE_Tokenizer::Preserve_Entry::start_, ACE_Tokenizer::Preserve_Entry::stop_, and ACE_Tokenizer::Preserve_Entry::strip_. Referenced by ACE_Process_Options::command_line_argv(). { if (preserves_index_ == MAX_PRESERVES) return -1; preserves_[preserves_index_].start_ = start; preserves_[preserves_index_].stop_ = stop; preserves_[preserves_index_].strip_ = strip; preserves_index_++; return 0; }

---

Member Data Documentation

ACE_TCHAR* ACE_Tokenizer::buffer_ [protected]

Definition at line 398 of file SString.h.

int ACE_Tokenizer::delimiter_index_ [protected]

Pointer to the next free space in delimiters_. Definition at line 458 of file SString.h. Referenced by delimiter(), delimiter_replace(), and is_delimiter().

Delimiter_Entry ACE_Tokenizer::delimiters_[MAX_DELIMITERS] [protected]

The tokenizer allows MAX_DELIMITERS number of delimiters. Definition at line 455 of file SString.h. Referenced by delimiter(), delimiter_replace(), and is_delimiter().

int ACE_Tokenizer::index_ [protected]

Definition at line 399 of file SString.h.

Preserve_Entry ACE_Tokenizer::preserves_[MAX_PRESERVES] [protected]

The application can specify MAX_PRESERVES preserve designators. Definition at line 427 of file SString.h. Referenced by is_preserve_designator(), and preserve_designators().

int ACE_Tokenizer::preserves_index_ [protected]

Pointer to the next free spot in preserves_. Definition at line 430 of file SString.h. Referenced by is_preserve_designator(), and preserve_designators().

---

The documentation for this class was generated from the following files:

• SString.h
• SString.cpp

---