[/ 
  (C) Copyright Edward Diener 2011-2015,2020
  Distributed under the Boost Software License, Version 1.0.
  (See accompanying file LICENSE_1_0.txt or copy at
  http://www.boost.org/LICENSE_1_0.txt).
]

[section:vmd_identifier Identifiers]

An identifier in VMD is either of two lower-level preprocessor possibilities:

* a preprocessing token 'identifier', which is essentially a sequence 
of alphanumeric characters and the underscore 
character with the first character not being a numeric character.
* a preprocessing token 'pp-number' that is an integral literal token.

Here are some examples:

 SOME_NAME
 _SOME_NAME
 SOME_123_NAME
 some_123_name
 sOMe_123_NAmE
 2367
 43e11
 0
 22
 654792
 0x1256

[heading Problem testing any identifier]

One of the difficulties with identifiers in preprocessor metaprogramming 
is safely testing for a particular one. VMD has a means of doing this within
a particular constraint for the characters that serve as the input to test.

The constraint is that the input must either be empty or that the beginning
input character, ignoring any whitespace, passed as the input to test must be either:

* an identifier character, ie. an alphanumeric or an underscore
* the left parenthesis of a tuple 

and if the first character is not the left parenthesis of a tuple
the remaining characters must be alphanumeric or an underscore until a space character
or end of input occurs

If this is not the case the behavior is undefined, and most likely
a preprocessing error will occur.

Given the input:

 's_anything' : can be tested
 'S_anything' : can be tested
 's_anYthiNg' : can be tested
 '_anything' : can be tested
 '_Anything' : can be tested
 '_anytHIng' : can be tested
 '24' : can be tested
 '245e2' : can be tested
 '(anything)' : can be tested, tuple
 '(anything) anything' : can be tested, tuple and further input
 'anything anything' : can be tested, identifier followed by space character
 
 '%_anything' : undefined behavior and most likely a preprocessing error due to the constraint
 '(_anything' : undefined behavior and most likely a preprocessing error due to the constraint, since a single '(' does not form a tuple
 '44.3' : undefined behavior and most likely a preprocessing error due to the constraint since '.' is not alphanumeric
 
[heading Identifying an identifier]

There are three levels of testing identifiers in VMD.

# You can test for any general identifier which matches the identifier syntax
# You can test for a specific identifier
# You can test that a specific identifier is equal to another specific identifier

[heading Testing for any general identifier]

The macro for testing for any identifier is called BOOST_VMD_IS_GENERAL_IDENTIFIER.

You invoke it with input which may be an identifier according to the rules given
above, and it returns 1 if the input has the form of an identifier and returns 0
if the input does not have the form of an identifier. Be aware that input which
does not follow the above constraints will probably lead to undefined behavior. Also
the input to BOOST_VMD_IS_GENERAL_IDENTIFIER is meant to be a single identifier and not a VMD sequence of preprocessor
tokens, else the return may be incorrect. Sample code:

 #include <boost/vmd/is_generaL_identifier.hpp>
 
 BOOST_VMD_IS_GENERAL_IDENTIFIER(some_input) // returns 1 if 'some_input' is in the form of an identifier, else returns 0
 
This form of an identifier which follows the identifier syntax is called a "general" identifier.
General identifiers can be tested successfully using the VMD specific macros but can not be tested
successfully using the VMD generic macros. For the latter you need specific identifiers.
 
[heading Testing for a specific identifier]

Although testing whether macro input is in the form of any general identifier given above may be useful
sometimes it is valuable to find out whether macro input is one of a number of
specific identifiers in which you are particularly interested. In order to do this
you first create a user-defined macro which 'registers' specific identifiers.
The user-defined macro is an object-like macro whose form is:

 #define BOOST_VMD_REGISTER_identifier (identifier)
 
where 'identifier' is a specific identifier we wish to identify. This is called in
VMD a registration macro.

The macro for testing for a specific identifier is called BOOST_VMD_IS_IDENTIFIER.

 #include <boost/vmd/is_identifier.hpp>
 
 #define BOOST_VMD_REGISTER_yellow (yellow)
 #define BOOST_VMD_REGISTER_green  (green)
 #define BOOST_VMD_REGISTER_blue   (blue)
 
 BOOST_VMD_IS_IDENTIFIER(some_input) // returns 1 if 'some_input' is 'yellow','green', or 'blue'
 BOOST_VMD_IS_IDENTIFIER(some_input) // returns 0 if 'some_input' is 'purple'

It is recommended that registration macros be created in a header file which
can be included before the end-user uses the identifier macros of VMD, but of course you can
create registration macros directly in a source file if you wish.
If a particular registration macro occurs more than once it is
not a preprocessing error, so duplicating a registration macro will not lead to any problems
since each registration macro of the same name will have the exact same object-like macro
expansion.
Within a given translation unit it could potentially happen
that registration macros have been included by header files which a particular end-user
of VMD has not created. This should also not lead to particular problems since registration
is a process for adding specific identifiers for any particular translation unit.

As we shall see further in the documentation only specific identifiers can be tested
for when we work with VMD generic macros and to parse VMD sequences, so having
an identifier which is not a specific identifier but only a general identifier has a limited use in VMD.

[heading Testing for identifier equality]

Although registering an identifier allows VMD to recognize the string of characters
as a specific VMD identifier when using the BOOST_VMD_IS_IDENTIFIER, the ability
to detect whether or not a specific identifier is exactly the same or not than another
specific identifier needs the end-user to define another macro:

 #define BOOST_VMD_DETECT_identifier_identifier
 
where 'identifier' is a specific identifier we wish to detect. This object-like 
macro must expand to no output.

Like the registration macro, multiple detection macros of the same identifier
in a translation unit does not cause a compiler problem since the exact same
object-like macro occurs.

The term for creating this macro is that we have potentially 'pre-detected'
the identifier and I will use the term pre-detected as the process of creating
the BOOST_VMD_DETECT macro and the term 'detect' as the ability to find out
if a specific identifier is the same as another specific identifier or not.

The ability to detect that a VMD identifier is a particular specific identifier is used
in VMD macros when preprocessor data is compared for equality/inequality using the
VMD macros [link variadic_macro_data.vmd_specific_generic.vmd_generic.vmd_equality BOOST_VMD_EQUAL and BOOST_VMD_NOT_EQUAL] 
as well as when we want to match an identifier against a specific set of other registered identifiers
using the macro [link variadic_macro_data.vmd_modifiers.vmd_modifiers_identifier.imid BOOST_VMD_IS_IDENTIFIER]
with more than the single variadic argument given above.

These situations will be explained later in the documentation in other topics
discussing VMD macro functionality. If the programmer never uses the functionality
which these situations encompass, ie. in which we match a specific identifier against another
to see if they are equal or not, there is no need to use pre-detection for a
registered identifier. However once you provide a registration macro in order to test
for a specific identifier in preprocessor input also providing a pre-detection macro for that identifier
costs very little in terms of macro notation and nothing in terms of preprocessor time. So the end-user
might want to add a pre-detection macro for a specific identifier to the same place or header
file where the corresponding registration macro is created, even if the end-user initially does
not think that he might want to test that identifier for equality with another identifier.

To sum up the three levels of identifier functionality:

* General identifiers can be used with the VMD specific macros and will return the correct result,
  but can not be used with VMD generic macros.
* Specific identifiers, ie. registered identifiers, can be used with VMD specific macros and
  nearly all VMD generic macros.
* Pre-detected specific identifiers can be used with all VMD macros, whether specific or generic.

[heading Parsing identifier constraints and undefined behavior]

The reason that the identifier constraints mentioned above exist is that the
technique for parsing identifiers, once it is determined that the input
being parsed is not empty or does not begin with a set of parentheses, uses preprocessor 
concatenation in its parsing. This technique involves the preprocessor '##' 
operator to concatenate input, and examine the results of that concatenation.
When preprocessor concatenation is used the result of the concatenation must
be a valid preprocessing token, else the behavior of the preprocessor is undefined.
In C++ 'undefined behavior' in general means that anything can happen. In practical
use when preprocessor concatenation does not produce a valid preprocessing token,
a compiler is most likely to generate a preprocessing error. If the compiler chooses
not to issue a preprocessing error when concatenation does not produce a valid preprocessor token
the outcome of the VMD macros will always be correct and parsing an identifier will fail.
But because the outcome is undefined behavior there is no absolute way that the programmer
can determine what the outcome will be when preprocessor concatenation is used and the input
being parsed contains does not meet the constraints for parsing an identifier
mentioned at the beginning of this topic.

In this documentation I will be using the abbreviation 'UB' as the shortened form
of 'undefined behavior' to denote the particular occurrence where VMD attempts to
parse preprocessor input using preprocessor concatenation and undefined behavior
will occur.

[heading Usage]

To use the BOOST_VMD_IS_GENERAL_IDENTIFIER macro and/or the BOOST_VMD_IS_IDENTIFIER
macro you can either include the general header:

 #include <boost/vmd/vmd.hpp>
 
or include the specific headers:

 #include <boost/vmd/is_general_identifier.hpp>
 #include <boost/vmd/is_identifier.hpp>

[endsect]
