man zoiddevel () - Development documentation for zoid
NAME
zoiddevel - Development documentation for zoid
DESCRIPTION
Debugging
If you switch on the global debug bit and the verbose bit both your input and debug information are echoed to STDERR. Thus by piping both STDOUT and STDERR to a file you get a detailed log of what is happening.
To set both bits type:
zoid$ set debug zoid$ set verbose
or start zoid with:
$ zoid -vD
You can also get debug information from just one module by setting a global variable CW$DEBUG in the module's namespace.
If you set the debug variable to a number higher than 1, you get a stack trace of that number of frames for each exception thrown using the CWerror() method.
Structure
class diagram
Zoidberg::Shell
/|\ Zoidberg::Contractor
| /|\ /|\
|__________________| |
| |
| Zoidberg::Job
Zoidberg /|\
|
|
Zoidberg::Job::Builtin
All other classes stand on their own (except perhaps for some internal helper classes).
Zoidberg::Shell is an interface class to be used in perl scripts like zoidrc. Zoidberg inheriting from it in order to make this interface also available through the CW$shell object in the eval scope and to plugins. The Zoidberg::Contractor class contains the code to manage Jobs. Because the Job class also inherits from it sub-jobs are possible.
object structure
main object class Zoidberg
|
|___ {eval} Zoidberg::Eval object
|___ {stringparser} Zoidberg::StringParser object
|
|___ {jobs} Array with Job objects
| |_ items of class Zoidberg::Job
|
|___ {settings} Settings and config hashes
|___ {objects} Plugin objects - tied Zoidberg::PluginHash
|___ {parser} Custom parsing - tied Zoidberg::DispatchTable
|___ {events} Event code refs - tied Zoidberg::DispatchTable
|___ {commands} Command code refs - tied Zoidberg::DispatchTable
|___ {aliases} Alias definitions
FIXME explanation
Process flow
start
|
| incomplete input - readmore
| .--------------------.
| | |
| | |
V V string | tree
input loop --------> parse string ------> execute tree
Z:main_loop() Z:shell_string() Z:C:shell_list()
readline |
A reincarnate |
| ,------> |
new | next chunk | |
input | of blocks | | logic chunk
| | |
| | V
'---------------------------------------+--- execute job
Z:J:exec()
Z: = Zoidberg:: Z:C: = Zoidberg::Contractor:: Z:J: = Zoidberg::Job::
The parse tree going to CWshell_list() is a simple parse tree consisting of scalars and scalar references (see FIXME below). This tree is processed in logic chunks, corresponding to one job each. The blocks references in a logic chunk are parsed by CWparse_block() just before executing. This is where the context is determined and things like expansions and subtitutions are done.
Parsing
FIXME put devel opts in scheme
string --> shell_string()
split script_gram
|
V
list --> shell_list() ... FIXME tell bout this format below
|
V
ref --> parse_block() ... FIXME tell bout possible block forms
|
V ------------.
|
parse_env() | if word block
|
split word_gram |
split redir_gram |
strip env declaration A |
| |
| <------------------'
V |
alias expansion -------' if multiple blocks result
custom filtering
decide context unless already known
- context on whole string like perl
- custom word lists
- default word context
|----> custom parser
|
|----> parse_perl() for perl context
|
| for word context ---------. if 'FIXME'
V |
|
parse_words() V
expansion ... FIXME list them in zoiduser
(globs)
A block that originates from CWshell() or CWZoidberg::Shell::AUTOLOAD is recognised as having already been split into words. This means the parsing of words, redirections and environment declarations is skipped. Only alias are checked so this interface is consistent in command names etc. with the commandline interface.
If you use CWbuiltin() or CWcommand(), these are understood as simple command with parsed words and effectively all parsing is skipped.
Parse tree
FIXME tell about the blocks used in lots of hooks
FIXME renice this stuff
This part describes the form of a parse tree as used between the various Zoidberg objects.
Example
# Commandline input:
$ ls -al | perl{ while (<STDIN>) { print $_ } }abc && echo done
# Would be parsed to:
[
[{context => 'CMD'}, qw/ls -al/],
[{context => 'PERL', opts => 'abc'}, q{ while (<STDIN>) { print $_ } } ],
'AND',
[{context => 'CMD'}, qw/echo done/]
]
# Commandline input:
$ cd .. && ls -al ; cp dus ~/tmp/ &
# Would be parsed to:
[
[{context => 'CMD'}, qw/cd ../],
'AND',
[{context => 'CMD'}, qw/ls -al/],
'EOS', # End Of Statement
[{context => 'CMD'}, qw{cp dus ~/tmp/}],
'EOS_BG' # background statement
]
# FIXME an example with redirections
There are a lot more meta fields like 'fork_job', FIXME where to read bout them?
Basics
A parse tree is an array consisting of blocks and tokens. A block can be any kind of shell code and is stored in a nested array. Blocks directly following each other are supposed to be a pipeline. A token is a delimiter between blocks.
The first field of a block is a hash which contains information about the block, all other field in a block make up the content. The most important information about a block is the context, which tells the parser how to execute the block. You are free to store all kinds of specific information in this first field, but some key names are reserved.
FIXME reserved meta fields
Pseudo parse trees
These are forms that can be used with the CWshell() function provided by Zoidberg::Shell. Just as by the real parse tree blocks of code are references and tokens are plain scalars. A block that is a scalar reference will be split into words and parsed completely (although still expected to be one block). A block that is an array reference will be considered to be completely parsed if the first element is a hash reference and the context defined, else it is considered a word list, possibly with meta data defined in the first element.
# for example "ls -al | perl{ while (<STDIN>) { print $_ } }abc && echo done"
# can be executed by calling :
shell(
[qw/ls -al/],
\'perl{ while (<STDIN>) { print $_ } }abc',
'AND'
[{context => 'CMD'}, qw/echo done/]
);
Using this kind of pseudo trees only makes sense if you are lazy or you don't know exactly what the command is but you have some clues you want to pass to the parser.
Built-in contexts
The contexts 'PERL', 'CMD' and 'SUBZ' are used by the built-in parser: 'PERL' flags code as perl code, 'CMD' as a system or built-in command and the 'SUBZ' context is used for sub-shells.
Parser hooks
The hash called 'parser' in the main object is a tied hash of the class Zoidberg::DispatchTable. This hash contains stacks of code refs to be called at certain parser stages.
Subroutines in these stacks should in general return a BLOCK (a array ref with a meta field and words) at success and CWundef otherwise.
- filter BLOCK
- Stack called before contexts are decided and before the block is splitted into words. Used to claim a block for a certain context by setting the 'context' field in the meta hash.
- word_expansion BLOCK
- Allows you to define a custom expansion routine.
- word_list BLOCK
- Stack with a dual function: used to claim a block for a certain context after word splitting, but also for listing completions for this context. If called in CWwantarray context should return a list of possible completions for the first word in BLOCK. If not in CWwantarray context can either return a block with the 'context' field set to claim this block for a certain context.
You can define a custom context by putting a sub-hash into the parser hash (using the context name as the key). See Custom contexts for the keys to be defined in such a sub-hash (note that some of the stacks are included there also - but these won't go in the sub-hash).
Settings
Notice that every plugin has it's own config hash in the settings hash, it is suggested that settings only affecting one plugin are placed in this sub-hash. This rule is broken for some settings required by the posix specification of the CWset built-in.
FIXME list these posix settings somwhere
Below are listed advanced settings only needed for development. Common settings can be found in zoiduser.
The parser settings can be set globally, but also per block. FIXME they should also be settable per mode/context
- split_script
- Split a command according to the script grammar.
- split_words
- Split a command into words.
- parse_fd
- Parse redirections.
- parse_env
- Parse environment declerations.
- parse_aliases
- Check for aliases.
- parse_def_contexts
- Use the default contexts PERL and CMD.
- expand_comm
- Expand commands in word contexts.
- expand_param
- Expand parameters in word contexts.
- expand_path
- Expand globs in word contexts.
- debug
- Turn on _all_ debug messages.
- plain_words
- This is not a global setting, but if used for a single block it is identical to turning off all parser settings except for 'parse_aliases'.
Events
The hash called 'events' in the main object is a tied hash of the class Zoidberg::DispatchTable. This hash contains stacks of code refs to be called when certain events happen.
Some events used:
FIXME more events
- beat
- Called every idle second when using Term::ReadLine::Zoid. Used for asynchronous stuff.
- newpwd
- Called if CW$ENV{PWD} has changed. The old pwd can be found in CW$ENV{OLDPWD}.
- prompt
- This event is called from the main loop just before respawning the prompt. It can be used to update routines.
- cmd
- This event is called after the input prompt with the commandline as the first argument. It is intended for interactive input only.
- envupdate
- This event is called after the spawning of every job. Like 'prompt' it can be used for update routines; since 'envupdate' is called more often then 'prompt', use 'prompt' if it makes no difference, 'envupdate' should only be used for atomic updates.
- loadrc
- Called after all the plugins are loaded, it triggers reading the rcfiles. Can be used by plugins to set hooks that depend on other plugins being loaded.
- exit
- Called just before CWround_up(), used as replacement for .bash_logout.
- set_*
- Whenever a setting is changed a event is called which is the name of the setting prefixed by 'set_' . Arguments are the new value and the old value of the setting. When the new value is 'undef' the setting was deleted. These events are generated by the Zoidberg::SettingHash class, which is used to tie the settings hash of the main Zoidberg object.
- plug_*
- unplug_*
- Called when a plugin is loaded or deleted.
Plugins
Zoid makes extensive use of plugins to perform tasks that are not considered the core functionality of a shell (and very little is). A plugin has three ways of communicating with the the shell and other plugins:
- builtins
- These are commands implemented in the shell to offer some function to both the user and other plugins.
- events
- A plugin can both trigger events ('broadcast') or listen to certain events. When an event is triggered all code 'listening' to it is executed, this is a one-to-many dispatcher.
- contexts
- Used to extend the code types zoid knows. By default only command syntax and perl are recognized, but a plugin might want to add other syntaxes. A context is typically used when it is unconvenient to define all commands the plugin can handle as builtins, or when a plugin want to handle the whole code block instead of the individual (possibly expanded) words. Also some methods are provided to hook into certain parser stages.
Some general tips for writing plugins:
- •
- See the Zoidberg::Utils namespace for common functions. Using these functions will make your plugin more coherent with the rest of the shell.
- •
- Try to use api's, not data structures; unless the structures are specified in the documentation they can change without notice. Api's are in general more stable.
- •
- Try to use AutoLoader if you have a lot of optional functionality. ( When using AutoLoader make sure your subs have names that are case-insensitive unique, else you'll run into problems on OS X. )
- •
- When you need hooks between two plugins, use a builtin command. Make sure to find a name for it that doesn't overload a normal system command. When you need various hooks, see if they can be combined in one builtin command using options (see Zoidberg::Utils::GetOpt). Also check if other shell implementations (like bash, tcsh or zsh) have builtins for similar funtions and try to make your interface consistent. When calling a builtin as a hook the most efficient way is to use the CW$shell->builtin($cmd, @args) method in a list context. Using this method will cause a lot of parser levels to be skipped, also the command won't be subject to things like the current command mode. Using list context makes the method return directly without bothering with overloaded objects.
PluginConf
A plugin configuration file can use the following keys:
- module
- The class to bless the plugin object in.
- config
- Hash with plugin specific config stuff. For plugins that inherit from Zoidberg::Fish this will automatically become the CW{config} attribute.
- commands
- Hash linking command names to subroutines. See Zoidberg::DispatchTable for the string format used in this hash.
- export
- Array with commands automatically linking to a subroutine of the same name in the plugin object.
- events
- Like CWcommands but used for events.
- import
- Like CWexport but used for events.
- settings
- Supply some default settings. Used for global settings, plugin settings should be in the 'config' hash.
- aliases
- Supply some default aliases.
- context
- Add context hooks.
If you want your plugin to be loaded as soon as possible, you can use the loadrc event to load your plugin. For plugins inheriting from Zoidberg::Fish CWloadrc => 'plug' will work.
Custom contexts
Typically you'll want to set a custom context from a plugin. To do this you either provide some context hooks in the PluginConf or you set a loadrc event to load your plugin and then setup the context parser and handlers routines from the plugin's CWinit() routine, using the CWadd_context() method in CWZoidberg::Fish. If you defined a word_list or filter hook it makes sense to use the second method. The plugin would be loaded anyway when the first command is parsed.
A context config hash can contain the following routines:
FIXME more verbose description
- word_list BLOCK
- If you want your context to work with words you should have this hook to recognize and list commands. On wantarray it should return a list of possible completions for the word in CW$$block[1]; else it should check whether the word is a recognized command and return true on success. In both cases it is also allowed to return a block ref, this is for the more advanced options.
- parser BLOCK
- Here you can set options like for example CWfork_job or CWno_words for your context.
- handler BLOCK
- This is the part where your command gets executed.
- completion_function WORD, LINE, START, END
- Hook used by the Intel plugin for custom completion.
- intel BLOCK
- Hook used by the Intel plugin for custom completion.
- filter BLOCK
- Hook to filter blocks before they are parsed, this can be used for example for custom redirection code. Be aware that the a block might be filtered twice with the same meta hash, the second time should reset any meta fields set by the first time.
- word_expansion BLOCK
- Hook extra expansions into the parser.
Hello World plugin
Here follows an example to create a simple plugin with the hello_world builtin.
First create a dir ~/.zoid/plugins/HelloWorld.
Next create a module, for example :
package HelloWorldZoid;
# Zoidberg::Fish is the base class for plugins use Zoidberg::Fish; our @ISA = qw/Zoidberg::Fish/;
# Zoidberg::Utils provides the output method use Zoidberg::Utils;
# no need for a constructor, bootstrap with init
sub init { $$_[0]{config}{string} ||= "Hello world !" }
# and in this sub we actually print the string
sub hello_world { output $$_[0]{config}{string} }
# this will be called when the plugin is unloaded
sub round_up { output "someone help me" }
1; # keep require happy
Save the module as ~/.zoid/plugins/HelloWorld/HelloWorldFish.pm.
Then create a config file, this is just a perl script returning a config hash.
{
module => q/HelloWorldFish/,
config => {
# This hash will be both accessible as $shell->{settings}{HelloWorld}
# and as $shell->HelloWorld->{config}
string => q/Hello cruel world !/
},
export => ['hello_world'],
}
Save the config file as ~/.zoid/plugins/HelloWorld/PluginConf.pl.
After restarting zoid, you should have a builtin command hello_world that prints the string Hello cruel world !, and an object called HelloWorld. Of course you guessed already that you can control the string that will be printed from the config file, and also that it defaults to Hello world !. On run-time this string is available under CW$shell->{settings}{HelloWorld}{string}, it can be changed at any time.
ENVIRONMENT
The following environment variables are used be Zoidberg.
- ZOIDPID
- Contains the process id of a parent zoidberg shell, intended to be used for an IPC mechanism.
- ZOIDREF
- In forked child processes this variable contains a stringyfied version of the current Zoidberg object in charge. The parent process has a global hash CW%Zoidberg::OBJECTS which maps these strings to the original references. It is intended that an IPC mechanism uses this hash to convert strings back to references. This value should correspond to the object stored in the global CW$Zoidberg::CURRENT at the time of forking. In secondary scripts it is better to use CWZoidberg::Shell->current(). TODO this mechanism seems to have some problems
- ZOIDCMD
- Contains the string that caused the current command to be executed. Can be used by a command to inspect its arguments as they were before path expansions etc..