Introduction:
-------------
The emugen tool is a tool to generate a wire protocol implementation
based on provided API. The tool generates c++ encoder code that takes
API calls and encodes them into the wire and decoder code that decodes
the wire stream and calls server matching API.
The emugen tool includes additional functionality that enables to
generate an wrapper library. The wrapper library provides entry points
for the specified API, where each entry routes the call via a dispatch
table. The dispatch table may be initialized as required by a specific application.
The following paragraphs includes the following:
* Wire Protocol Description
* Input files description & format
* Generated code description.
Note: In this document, the caller is referred to as Encoder or Client
and the callee is referred to as the Decoder or Server. These terms
are used interchangeably by the context.
Wire Protocol packet structure:
-------------------------------
A general Encoder->Decoder packet is structured as following:
struct Packet {
unsigned int opcode;
unsigned int packet_len;
… parameter 1
… parameter 2
};
A general Decoder->Encoder reply is expected to be received in the
context of the ‘call’ that triggered it, thus it includes the reply
data only, with no context headers. In precise term terms, a reply
packet will look like:
struct {
...// reply data
};
consider the following function call:
int foo(int p1, short s1)
will be encoded into :
{
101, // foo opcode
14, // sizeof(opcode) + sizeof(packet_len) + sizeof(int) + sizeof(short)
p1, // 4 bytes
s1 // 2 bytes
}
Since ‘foo’ returns value, the caller is expected to read back the return packet from the server->client stream. The return value in this example is in thus the return packet is:
{
int retval;
}
Pointer decoding:
----------------
The wire protocol also allows exchanging of pointer data
(arrays). Pointers are defined with directions:
in : Data is sent from the caller to the calle
out: Data is sent from the callee to the caller
in_out: data is sent from the caller and return in place.
‘in’ and ‘in_out’ encoded with their len:
{
unsinged int pointer_data_len;
unsigned char data[pointer_data_len];… // pointer data
}
‘out’ pointers are encoded by data length only:
{
unsigned int pointer_data_len;
}
‘out’ and ‘in_out’ pointer’s data is returned in the return
packet. For example, consider the following call:
int foo(int n, int *ptr); // assume that ‘data’ is in_out pointer which contains ‘n’ ints
The caller packet will have the following form:
{
101, // foo opcode
xx, sizeof(opcode) + sizeof(datalen) + sizeof(int) + sizeof(unsigned int) + n * sizeof(int);
n, // the n parameter
n * sizeof(int), // size of the data in ptr
… // n* sizeof(int) bytes
}
The return packet is;
{
…. // n * sizeof(int) bytes of data return in ptr
retval // sizeof(int) - the return value of the function;
}
Endianess
---------
The Wire protocol is designed to impose minimum overhead on the client
side. Thus, the data endianness that is sent across the wire is
determined by the ‘client’ side. It is up to the server side to
determine the client endianess and marshal the packets as required.
Emugen input files - protocol specification
-------------------------------------------
The protocol generated by emugen consists of two input files:
1. basename.in - A sepcification of the protocol RPC procedures. This
part of the specification is expected to be generated automatically
from c/c++ header files or similar.
‘basename’ is the basename for the protocol and will be used to prefix
the files that are generated for this protocol. A line in the .in
file has the following format:
[prefix](retvalType, FuncName, <param type> [param name],...)
where
retvalType - The function return value type
FuncName - function name
<param type> mandatory parameter type
[param name] - optional parameter name
Examples:
GL_ENTRY(void, glVertex1f, float v)
XXX(int *, foo, int n, float, short)
XXX(void, glFlush, void)
Note: Empty lines in the file are ignored. A line starts with # is a comment
2. basename.attrib - Attributes information of the API.
This file includes additional flags, pointers datalen information and
global attributes of the protocol. For uptodate format of the file,
please refer to the specification file in the project source
tree. The format of the .attrib file is described below.
3. basename.types - Types information
This files describes the types that are described by the API. A type
is defined as follows:
<type name> <size in bits> <print format string> <is a pointer? true|false>
where:
<type name> is the name of the type as described in the API
<size in bits> 0, 8, 16, 32 sizes are accepted
<print format string> a string to format the value of the type, as
acceted by printf(3)
<is pointer?> true or false string species whether the type should be
treated as a pointer.
example:
GLint 32 %d false
GLint* 32 %p true
GLptr 32 %p true
Encoder generated code files
----------------------------
In order to generate the encoder files, one should run the ‘emugen’
tool as follows:
emugen -i <input directory> -E <encoder files output directory> <basename>
where:
<input directory> containes the api specification files (basename.in + basename.attrib)
<encoder directory> - a directory name to generate the encoder output files
basename - The basename for the api.
Assuming the basename is ‘api’, The following files are generated:
api_opcodes.h - defines the protocol opcodes. The first opcode value
is 0, unless defined otherwise in the .attrib file
api_entry.cpp - defines entry points for the functions that are
defined by the protocol. this File also includes a function call
‘setContextAccessor(void *(*f)()). This function should be used to
provide a callback function that is used by the functions to access
the encoder context. For example, such callback could fetch the
context from a Thread Local Storage (TLS) location.
api_client_proc.h - type defintions for the protocol procedures.
api_client_context.h - defines the client side dispatch table data
structure that stores the encoding functions. This data structure also
includes ‘accessors’ methods such that library user can override
default entries for special case handling.
api_client_context.cpp - defines an initialization function for
dispatch table
api_enc.h - This header file defines the encoder data strcuture. The
encoder data structure inherits its functionality from the
‘client_context’ class above and adds encoding and streaming
functionality.
api_enc.cpp - Encoder implementation.
Decoder generated files
-----------------------
In order to generate the decoder files, one should run the ‘emugen’
tool as follows:
emugen -i <input directory> -D <decoder files output directory> basename
where:
<input directory> containes the api specification files (basename.in + basename.attrib)
<decoder directory> - a directory name to generate the decoder output files
basename - The basename for the api.
With resepct to the example above, Emugen will generate the following
files:
api_opcodes.h - Protocol opcodes
api_server_proc.h - type definitions for the server side procedures
api_server_context.h - dispatch table the decoder functions
api_server_context.cpp - dispatch table initialization function
api_dec.h - Decoder header file
api_dec.cpp - Decoder implementation. In addtion, this file includes
an intiailization function that uses a user provided callback to
initialize the API server implementation. An example for such
initialization is loading a set of functions from a shared library
module.
Wrapper generated files
-----------------------
In order to generate a wrapper library files, one should run the
'emugen' tool as follows:
emugen -i <input directory> -W <wrapper files output directory> basename
where:
<input directory> containes the api specification files (basename.in + basename.attrib)
<wrapper directory> - a directory name to generate the wrapper output files
basename - The basename for the api.
With resepct to the example above, Emugen will generate the following
files:
api_wrapper_proc.h - type definitions for the wrapper procedures
api_wrapper_context.h - dispatch table the wrapper functions
api_wrapper_context.cpp - dispatch table initialization function
api_wrapper_entry.cpp - entry points for the API
.attrib file format description:
-------------------------------
The .attrib file is an input file to emugen and is used to provide
additional information that is required for the code generation.
The file format is as follows:
a line that starts with # is ignored (comment)
a empty line just whitespace of (" " "\t" "\n") is ignored.
The file is divided into 'sections', each describes a specific API
function call. A section starts with the name of the function in
column 0.
A section that starts with the reserved word 'GLOBAL' provides global
attributes.
below are few sections examples:
GLOBAL
encoder_headers string.h kuku.h
glVertex3fv
len data (size)
glTexImage2D
len pixels (pixels == NULL? 0 : (format_pixel_size(internalformat) * width * height * type_size(type)))
Global section flags description:
base_opcode
set the base opcode value for this api
format: base_opcode 100
encoder_headers
a list of headers that will be included in the encoder header file
format: encoder_headers <stdio.h> "kuku.h"
client_context_headers
a list of headers that will be included in the client context header file
format: client_context_headers <stdio.h> "kuku.h"
decoder_headers
a list of headers that will be included in the decoder header file
format: decoder_headers <stdio.h> "kuku.h"
server_context_headers
a list of headers that will be included in the server context header file
format: server_context_headers <stdio.h> "kuku.h"
Entry point flags description:
len
desciption : provide an expression to calcualte an expression data len
format: len <var name> <c expression that calcluates the data len>
custom_pack
description: provide an expression to pack data into the stream.
format: custom_pack <var name> <c++ expression that pack data from var into the stream>
The stream is represented by a (unsigned char *)ptr. The expression may also refer
to other function parameters. In addition, the expression may refer to 'void *self' which
is the encoding context as provided by the caller.
dir
description : set a pointer direction (in - for data that goes
to the codec, out from data that returns from the codec.
format: dir <varname> <[in | out | inout]>
var_flag
description : set variable flags
format: var_flag <varname> < nullAllowed | isLarge | ... >
nullAllowed -> for pointer variables, indicates that NULL is a valid value
isLarge -> for pointer variables, indicates that the data should be sent without an intermediate copy
flag
description: set entry point flag;
format: flag < unsupported | ... >
supported flags are:
unsupported - The encoder side implementation is pointed to "unsuppored reporting function".
custom_decoder - The decoder is expected to be provided with
custom implementation. The call to the
deocder function includes a pointer to the
context
not_api - the function is not native gl api