Initial revision

darcs-hash:20050920132639-ac50b-fa3b476891e1f5f67207cf4cc7bf623834cc5edc.gz

ChangeLog

@@ -0,0 +1,944 @@
+2005-09-19  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* env_universal.c, env_universal_common.c (barrier, callback, create_message, parse_message): Add support for blocking a client until all current messages from the client and corresponding replies  have been transmitted.
+
+	* env_universal.c (env_universal_read_all): Avoid recursive get_socket issues
+
+	* fish_pager.c (main, init, destroy): Return unused keypresses on pager exit
+	* reader.c (run_pager): Return unused keypresses on pager exit
+
+	* fish_pager.c (completion_try_print, main): Use alternate screen for interactive pager when available. 
+
+	* reader.c (reader_readline): Quit searching on escape
+
+
+2005-09-17  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* init/fish.in: Revert DISPLAY to non-array
+
+	* doc_src/doc.hdr: Documentation cleanup
+
+
+2005-09-16  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* builtin.c (builtin_status): Added the status builtin, for checking the fishprogram status
+
+	* doc_src/doc.hdr, init/fish, init/fish_function.fish, init/fish_complete.fish, init/fish_interactive.fish: Switch nomencalature: from subshell to command substitution
+
+	* doc_src/doc.hdr: Update documentation on variables
+	* doc_src/doc.hdr: Update documentation on commands and builtins
+	* doc_src/doc.hdr: Update documentation on completions
+
+	* init/fish_function.fish: help command now shows help for non-builtin commands	
+	* init/fish_function.fish: help command now searches man-pages
+
+	* env_universal.c, env_universal_common.c, fishd.c, env.c, builtin_set.c: Add support for universal variables
+
+	* env.c: Setting a local variable does not erase a global variable with the same name
+
+
+2005-09-13  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* exec.c (handle_new_child): Only call tcsetpgrp after creating new group
+
+	* exec.c, proc.c (setup_child_process, internal_exec_helper, handle_new_child, job_continue): Do not put block commands into their own groups
+
+
+2005-09-06  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* expand.c (tilde_expand): ~ should expand to $HOME, not getpwuid(getuid())->pw_dir
+
+2005-09-05  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* parser.c, builtin.c, Makefile.in (parse_job, builtin_not, builtin_run, builtin_get_desc): Add new 'not' builtin to negate commands
+
+	* parser.c, builtin.c, Makefile.in (parse_job, builtin_begin, builtin_run, builtin_get_desc): Add new 'begin' builtin to create unconditional block
+
+
+2005-09-01  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* complete.c (complete_load): Reload completions from file if completion file changes
+
+	* init/fish_functions: Added a 'type' shellscript function, with the same functionality as the bash builtin with the same name, and additional support for gnu style long options
+
+
+2005-08-31  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* init/completions/modprobe.fish: Add completions for modprobe command
+	* init/completions/service.fish: Add completions for service command
+	* init/completions/man.fish: Manual page completion now honors section specification
+	
+	* builtin.c, exec.c, proc.c (builtin_run, exec, job_continue): Builtins now follow the same rules w.r.t. setting the status variable as all other commands
+
+	
+2005-09-12  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.13.4
+	
+	* common.c (block, unblock, fgetws2): Smart blocking support to avoid excessive calls to signal blocking functions
+
+	* complete.c (complete): Fix infinite loop when completing string 'command '
+	
+
+2005-09-07  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.13.3
+	
+	* init/fish.in, init/fish_function.fish: Actually apply Netocrats patch, and not a broken patch causing random error messages
+
+2005-09-06  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.13.2
+
+2005-09-05  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* builtin.c (builtin_source): Fix crash when sourcing non-existant file
+
+	* init/fish.in: Only call unicode_start if in unicode locale
+	
+2005-09-05  Netocrat <netocrat@dodo.com.au>
+
+	* init/fish.in: Only set LANG in login shells
+	* init/fish_function (prompt_pwd): Don't use ellipsis character when not in unicode mode
+	
+2005-08-28  James Vega <jamessan@jamessan.com>
+
+	* function.c: Add missing #include "intern.h"
+
+	* env.c (env_set): GCC 4.0 workaround
+	
+2005-08-31  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* init/completions/function.fish: Some completions for function accidentally targeted the functions builtin. (Note the extra 's')
+	
+2005-08-31  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.13.1
+
+2005-08-31  Netocrat <netocrat@dodo.com.au>
+
+	* reader.c (write_prompt): Fix prompt flickering issue
+
+2005-08-31  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* exec.c (internal_exec_helper): Function blocks should not be marked as subshells
+	* exec.c (setup_child_process): Unblock childs signal handlers, fixes script bug
+
+2005-08-28  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.13.0
+
+2005-08-26  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* init/fish_inputrc: Remove the lame replace-commandline feature
+
+	* init/fish_inputrc: Remove the lame explain feature
+	
+	
+2005-08-25  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* proc.c (job_do_notifications): Do not notify user about status changed in subshell jobs
+
+	* init/completions/rpm.fish: Added package completion functionality
+
+	* init/completions/yum.fish: Added completions for yum command
+
+
+2005-08-24  Horea Haitonic <horeah@gmail.com>
+
+	* init/completions/diff.fish: Updated completions for diff command
+	
+	
+2005-08-24  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* util.c (sb_printf): Added sb_printf function for formated output into string buffer
+	* util.c (sb_append_in, sb_append_float): Dropped various stringbuffer functions in favour of sb_printf
+
+	* common.c (debug): Added function for making debugging messages
+
+	* builtin.c, exec.c, parser.c: Error reporting cleanups
+
+	* reader.c (reader_write_title): Make console detection slightly less lame
+
+
+2005-08-22  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* parser.c (parse_job): Make implicit cd use wrapper function around cd if available
+
+	* parser.c (parser_push_block): Fixed bug causing some global variables to be invisible inside functions
+
+	* env.c (env_push_block, env_pop_block, env_set, local_Scope_exports): Keep track of exported variables in scope to decrease amount of times we have to reexport variable values
+
+	* init/fish.in: Make sure the installation directory is added to the PATH variable, fixes issues where fish won't find it's various subcommands when installed in uncommon locations
+
+2005-08-22  Horea Haitonic <horeah@gmail.com>
+
+	* builtin.c (builtin_cd): Remove support for 'cd -' from cd builtin
+
+	* init/fish_function.fish(cd): Add support for 'cd -' to cd shellfunction wrapper
+	* init/fish_function.fish(cd): Add support for directory history
+	* init/fish_function.fish(pushd,popd): Add support for directory stack
+
+	
+2005-08-20  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* proc.c, reader.c (job_continue,set_signal_handlers): Make non-interactive mode use signals for handling children
+	* proc.c (job_continue): Use waitpid when there are no IO_BUFFER redirections
+	
+2005-08-19  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* exec.c (exec, launch_process, handle_child_process, setup_child_process): Clean up exec.c, add buffering for functions and blocks.
+	* parser.c (parser_push_block): Fix bug causing while loops in functions to be evaluated when the function was defined
+
+	* reader.c (handle_winch, check_winch): window size environment variables where erroneously set by signal handlers, causing potential crash.
+
+	* init/fish (contains): Made 'contains' much, much faster
+
+	* exec.c (exec): Fix bug causing exec builtin to crash the shell
+
+	
+2005-08-18  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* wildcard.c (get_desc): Re-add stat-call to command completion, but make sure only files that match the completion are stated.
+
+	* proc.c (io_add): io_add can now add together two chains of io redirections, which is needed when merging block-level and job-level redirections. Lack of this functionality caused a bug when doing complex io redirections.
+
+		
+2005-08-18  Jan Fader  <jan.fader@web.de>
+
+       * INSTALL: Changed the Chapter over "Local Install Procedure"
+
+       * configure.ac: Fixed not working installation to $HOME
+
+       * Makefile.in: Make echo at the end of the install target not verbose
+
+
+2005-08-16  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* init/completions/apropos.fish: Improve completions for aprops
+
+	* parser.c: Added profiling support. Set the PROFILE macro in parser.c to non-zero to enable.
+	
+2005-08-15  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* init/completions/wget.fish: Add completions for wget command
+
+	* exec.c (exec_subshell): Remove null entries caused by invalid unicode
+	
+2005-08-14  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* builtin_set.c (builtin_set): Fixed bug causing bug when changing variable scope without changing value
+
+	* exec.c, proc.c, parser.c: Added support for piping output of blocks
+	
+2005-08-12  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* complete.c (complete_cmd_desc): New implementation of command description lookup, much faster!
+
+	* wildcard.c (get_desc): Avoid calling stat during command completion for performance reasons. Command filesizes are no longer printed.
+	
+	* wutil.c (wgetcwd, wchdir): Add more wrapper functions
+	* builtin.c (builtin_cd): Use new wrappers in wutil
+
+	* wutil.c (wutil_wcs2str): Make convertions use internal buffer for better performance
+
+	* complete.c (complete_load): Add support for dynamic completion loading
+	* Makefile.in: Add build support for dynamic completion loading
+	* init/fish_complete: Split into multiple dynamically loaded parts
+	* builtin.c (builtin_complete): Add -y switch for requesting the loading of completions for a command
+	
+	* env.c: Decrease the amount of exported environment variable recalculations
+
+	* reader.c, builtin.c (do_exit, builtin_exit): Fixed bug causing fish_on_exit function not to be evaluated in very rare cases
+	
+2005-08-11  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* builtin_commandline.c (builtin_commandline): Split of to separate file
+	* builtin_commandline.c (builtin_commandline): Make the switches more logical and orthogonal
+
+	* builtin_commandline.c (builtin_commandline): Split of duplicate code to separate functions
+
+	* reader.c (reader_current_token_extent): Fix bug when a short string token follows a pipe or semicolon causing incorrect token identification
+
+	* reader.c (reader_current_job_extent, reader_current_process_extent): Add new extent computation commands
+	
+	
+2005-08-09  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* builtin.c (builtin_commandline): Add support for inserting readline functions from a command
+
+	* init/fish_inputrc: Change default behaviour of \C-d to __fish_delete_or_exit
+
+	* expand.c (expand_backslash): Add support for C-style  character escapes for unicode, as well as octal and hexadecimal ascii, i.e. \uxxxx, \Uxxxxxxxx, \xxx and \oooo.
+
+	* init/fish_complete.fish: Added skin-specific completions for Valgrind
+
+	* complete.c (complete_param): Fixed bug causing some completions to be omitted
+
+	* init/fish_complete.fish: Added completions for the commandline builtin
+
+2005-08-09  Horea Haitonic <horeah@gmail.com>
+
+	* builtin_set.c (update_values): Improve behavior in assigning array-slices with improper number of arguments
+
+	
+2005-08-08  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* exec.c (launch_process): Fix bug on redirection to self, i.e. >&1.
+
+	* env.c (env_set): Fix scoping misfeature causing improper scoping for exported variables
+
+	* env.c (env_set): Make it possible to explicitly unexport variables
+	* builtin_set.c (builtin_set): Add unexport switch to set builtin
+
+	* main.c (main): Fix broken login shell detection
+
+	
+2005-08-08  Horea Haitonic <horeah@gmail.com>
+	
+	* builtin_set.c: Completely new implementation of the 'set' builtin, supporting erasing sets of array elements.
+
+
+2005-08-05  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* intern.c (intern, intern_free_all): Add string pooling library, decreases memory usage by ~40kB
+
+	* exec.c proc.c (exec_subshell, ss_create, ss_wait, ss_clear, read_subshell): Remove old (forking) subshell implementation
+
+	* exec.c (exec): Make sure builtins don't fork when buffering output (slight performance increase, larger when using Valgrind)
+
+	* init/fish_complete.fish (__fish_complete_cd): Make sure we don't actually change the directory
+
+	* builtin.c (builtin_cd): Make sure OLDPWD is not changed when using cd in subshells
+
+	* builtin.c (builtin_read): Fixed bug causing random breakage
+
+	
+2005-08-04  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* complete.c (complete_add, complete_param): Add support for conditional command completions
+
+	* builtin.c (builtin_complete): Add switches to the complete builtin to support conditional command completion
+
+	* util.c (al_set): Make sure unused array_list_t elements are zeroed
+
+
+2005-08-04  Horea Haitonic <horeah@gmail.com>
+	
+	* util.c (al_push): Set new list size on resize
+
+
+2005-07-30  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* exec.c, proc.c (exec_subshell): Execute subshells in main process
+	
+	
+2005-07-28  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.12.1
+	
+	* expand.c (expand_file_completion): Removed unneeded function, the program should call expand_string directly
+	* expand.c (expand_string): Remove unneeded wrapper, expand_string is now the main function
+
+	* complete.c (complete_param_expand): Renamed complete_param_file to complete_param_expand, since it does completions for more than just filenames
+
+	* complete.c (complete): Make sure process completion is always performed
+	
+	
+2005-07-27  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* init/fish_complete.fish: A common group completion function, used by chgrp and a few others
+	* init/fish_complete.fish: Tweak kill completion to work on non-Debian systems
+	
+2005-07-26 Horea Haitonic <horeah@gmail.com>
+
+	* init/fish_complete.fish: Add completions for kill, zip and diff commands.
+	* init/fish_complete.fish: A common process completion function, used by ps and top
+	
+2005-07-17  James Vega <jamessan@jamessan.com>
+
+	* doc_src/set_color.c, doc_src/tokenize.c, doc_src/mimedb.c: Add missing #includes
+	
+2005-07-17  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* parser.c (parser_is_block, parser_is_subcommand, parsert_is_reserved): Changed name of functions parser_block_add, parser_subcommand and parser_reserved
+
+	* tokenizer.c, tokenizer.h (tok_init, tok_next): Changed return type to void
+
+	* doc_src/complete.txt: Added documentation for dynamic completion descriptions
+
+	* parser.c (parse_job, eval_job, eval, parse_job_main_loop): Remove unused tokenizer stacking support
+
+	* reader.c: Code cleanup, API documentation additions.
+	
+2005-07-16  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* fish_tests.c: Add Library test functions
+
+	* configure.ac: Handle installation to subdirectories of $HOME properly
+
+	* tokenizer.c (tok_init): Add checks for invalid input 
+	
+	* parser.c (parser_test): Add checks for invalid input 
+
+2005-07-15  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.12.0
+	
+2005-07-15 Horea Haitonic <horeah@gmail.com>
+
+	* init/fish_complete.fish: Updated make completions to make them more robust
+
+2005-07-14  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* complete.c (complete_init, complete_destroy): Remove table with descriptions for executables
+
+	* complete.c (complete): Fixed bug where fish failed to find the command to complete
+
+	* complete.c (complete_add): Fixed missing initialization	
+
+	* env.c (env_init): Removed remnant of $random variable
+
+	* init/fish_complete.fish: Fixed a few typos, misspelled words, etc
+	* init/fish_complete.fish: Make sure -x flag is set properly 
+	* init/fish_complete.fish(__fish_print_users): Added generic username completion
+	
+	* reader.c (completion_print): Fixed width miscalculation bug in completion listing code
+	
+2005-07-13  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* common.c (str2wcs): Make the maximum error report count runtime configurable
+	* complete.c (complete): Do not report string convertion errors in completion mode
+
+	* builtin.c, builtin.h, exec.c (builtin_exec, builtin_run): Renamed builtin_exec to builtin_run to avoid nameclash when adding the builtin 'exec'
+	* builtin.c, parser.c, exec.c (builtin_exec, builtin_run, exec, parse_job): Add builtin 'exec'
+
+2005-07-08  James Vega <jamessan@jamessan.com>
+
+	* reader.c (reader_sanity_check): Remove useless sanity checks
+	
+2005-07-08  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* autoconf.ac: Fixed docdir bug
+
+	* init/fish_funcion.fish (help): Fixed bug causing fish not to show right section for builtins
+	
+2005-07-07  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* wildcard.c(wildcard_expand): File completion now includes file size
+	
+	* init/fish_complete.fish: Tweaked completions
+
+2005-07-07  Yongjian Xu <i3dmaster@gmail.com>
+
+	* init/fish_complete.fish: Added loads of new completions
+
+2005-07-05  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* parser.c (parser_reserved): Added reserved words 'end', 'else', 'case', 'builtin' and 'command'
+	
+2005-07-04  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* builtin.c (builtin_commandline): Add token handling functionality to commandline builtin
+	* parser.c, complete.c (parser_check_command, complete): Remove token handling functionality through environment variables - replaced by commandline builtin
+
+	* init/fish_functions.fish (contains,_contains_help): Added a contains function, for checking if a key is in a set
+	
+2005-07-01  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* reader.c (reader_current_token_extent, reader_current_subshell_extent): Factor out token handling code
+	
+	* reader.c (reader_readline, handle_token_history): Add token search functionality
+
+2005-06-27  James Vega <jamessan@jamessan.com>
+
+	* input.c (add_emacs_bindings): Fix spelling error causing broken keybindings
+	
+2005-06-27  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* complete.c (complete_add,complete_parm): Add support for old and gnu style long option completion for the same command
+
+	* gen_hdr2.c (main): Added missing return statement
+
+	
+2005-06-24  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* init/fish_complete.fish (__fish_complete_suffix): Add general purpose file completion function
+	* init/fish_complete.fish (__fish_complete_directory): Add general purpose directory completion function
+	* init/fish_complete.fish (__fish_rpm_complete): Add rpm completions
+	* init/fish_complete.fish (__fish_cd_complete): Make cd completion it's own function 
+
+	* function.c (function_get_names,get_names_internal): Add support for hidden functions
+	* builtin.c (builtin_function,builtin_functions): Add support for hidden functions
+	* complete.c (complete_cmd): Add support for hidden functions
+
+	* init/fish_complete.fish: Update completions for set_color
+	
+2005-06-19  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* input.c(input_init,add_emacs_bindings,add_common_bindings,add_vi_bindings): Add initial vi-mode implementation
+
+2005-06-17  James Vega <jamessan@jamessan.com>
+
+	* Makefile.in: Add -fno-strict-aliasing compiler flag
+
+2005-06-16  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.11.1!
+
+	* doc_src/doc.hdr: Small documentation tweak to work around a bug in Doxygen 1.4.3
+	
+	* env.c (env_init): Make HOME variable writeable by root
+	
+	* reader.c, configure.ac (writembs): Ugly kludge to make powerpc work
+	
+	* parser.c (parser_reserved): Add function for checking if a word is reserved
+	* parser.c (parser_test): Made validator use parser_reserved function
+	* builtin.c(builtin_function): Made function definitions respect reserved words
+
+2005-06-14  Tom Arnett <tarnett@mailshell.com>
+
+	* init/fish_function.fish (prompt_pwd): Corrected the escape for the ellipsis character
+	
+2005-06-14  James Vega <jamessan@jamessan.com>
+
+	* builtin.c (builtin_cd): Add support for '-' argument, which performs a cd to the previous working directory.
+	* builtin.c (set_pwd): Extract out common functionality for setting pwd.
+
+2005-06-15  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* proc.c (handle_child_status): Remove debug message
+	
+	* init/fish_interactive.fish, highlight.h: Changed the color environment variables to lower case, since they are not global variables
+
+	* doc_src/doc.hdr: Updated sections on fish_prompt and fish_title to reflect that they are now functions, and not environment variables
+
+2005-06-14  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.11.0!
+	
+	* common.c (str2wcs, error_reset): Add an upper limit to the number of minor errors to report for a single command. 
+
+2005-06-13  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* configure.ac: Make default configuration directory to always be /etc, never $PREFIX/etc
+
+	* reader.c(write_prompt): Make fish_prompt a function
+	* init/fish_interactive.fish(fish_prompt): Make fish_prompt a function
+	* reader.c(write_title): Make fish_title a function
+
+	* init/fish: Make sure the PATH is sane
+
+	* builtin.c(builtin_builtin):Add support for -n option
+	* builtin.c(builtin_set):Add support for -n option
+	* builtin.c(builtin_functions):Add support for -n option
+
+	* init/fish_complete.fish: Add completions for scp command
+
+	
+2005-06-11  Michael Wardle <michael@endbracket.net>
+
+	* input.c (input_parse_inputrc_line): tilde expand parameter to 'include'
+
+	
+2005-06-11  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* expand.c (expand_brackets): Improve tab completion in bracket expantion context
+	* expand.c (expand_brackets): Fix handling of the comma character in bracket expantion
+
+	
+2005-06-10  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* init/fish_complete.fish: Additional completions for mplayer
+	* init/fish_complete.fish: Completion information for the date command
+
+	* expand.h, wildcard.h: Moved various constants from wildcard.h to expand.h
+	
+
+2005-06-10   Michael Wardle <michael@endbracket.net>
+
+	* doc_src/doc.hdr: Spelling correction
+	* doc_src/doc.hdr: Fixed description of \b escape
+
+	* init/fish_complete.fish: Drop error messages on missing files when completing ssh hosts
+
+	
+2005-06-08  Bram Senders <bram@luon.net>
+	
+	* init/fish_function.fish (help): Added www-browser and x-www-browser as one of the possible help browsers. 
+
+	
+2005-06-08  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* main.c (main): Event hooks are now functions and not environment variables
+	* reader.c (read_i): Event hooks are now functions and not environment variables
+	* reader.c (read_i): Added fish_on_exec hook
+
+	* builtin.c (builtin_functions): functions builtin can now be used to redefine the description of functions
+
+	* doc_src/doc.hdr: Changelog now lives in it's own file
+	* ChangeLog: Changelog now lives in it's own file
+
+	
+2005-06-08  James Vega <jamessan@jamessan.com>
+
+	* main.c (main): Add support for the -i option (force interactive mode)
+
+	* Various files: A large number of code cleanups, removing warnings
+
+	* Various files: Insurmountable number of spelling fixes
+
+	
+2005-06-08  Jan Fader <jan.fader@web.de>
+
+	* Makefile.in: Remove install_local option and replace it with correct handling of configuration files
+	* configure.ac: Remove install_local option and replace it with correct handling of configuration files
+
+	
+2005-06-07  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* parser.c (skipped_exec): Fixed crash bug when using a switch statement in a function
+
+	* parser.c (parser_test): Fixed unhelpful error messages when evaluating a script
+
+	* init/fish_function.fish (help): Fixed text-based browsers going to the background even if they are console based
+	* init/fish_function.fish (vared): Added support for a help option
+
+
+2005-06-01  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.10.1!
+
+	* expand.c: %self now expands to the shells own pid
+
+	* complete.c: Fixed bug with completion descriptions for completion of command names containing spaces
+
+	* init/fish_function.fish: The help command was broken in the previous release
+
+	* init/fish_complete.fish:  Fixed bug in tab completions for cd command. The 'new and improved' completion script in 1.10 was a bit buggy, it seems...
+
+	* expand.c: Using double dollars for creating map variables was accidentally broken. This meant among other things that the vared command didn't work. This bug was reported by Jan Fader
+
+	* init/fish_functions.fish: Improved error handling in the vared function
+	* init/fish_functions.fish: The vared command did not work when given the name of an undefined variable
+
+	* configure.ac: There where multiple issues with fish when compiled witout the xsel command. Jan Fader pointed these bugs out and provided patches for some of them.
+
+	* Makefile.in: Fixed build script bug causing the integrated help message for the commandline builtin to dissapear
+
+	* reader.c: Fixed rendering bug causing incorrect prompt after executing some keyboard shortcuts
+
+
+2005-05-28  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.10!
+
+	* builtin.c: Added a return builtin.
+
+	* builtin.c, function.c, complete.c: Functions can now have meaningful completion descriptions.
+	* builtin.c, complete.c: Builtin commands now have meaningful completion descriptions.
+	* A more robust method for finding the help documentation is used.
+	* If the current directory is very long, the prompt is ellipsised.
+	* Long commands are ellipsised instead of overflowing to the next line.
+	* Improved completions for cd, uniq, gcc and mplayer.
+	* Much improved performance in the apparently common case of holding down the enter key for a longer stretch of time.
+	* Fixed bug where the PWD variable was set incorrect on startup.
+	* Fixed bug where appending output to a non-existing file would fail.
+
+2005-05-24  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.9.2
+
+	* Error message from the cd command was missing a newline, which caused it to be erased by the prompt.
+	* Help browser is now put to the background if it is known to be graphical. Mikael Rasmussen suggested this.
+	* Added hook fish_on_return, specifying command to be called when fish returns from evaluating a command.
+	* Changed names of variables that are not exported to lower case.
+	* make uninstall missed a few files. This is now fixed, thanks to Jan Fader.
+	* Specifying a prefix to configure now works. Many people sent in patches for this, including Valient Gough, James Vega and Jan Fader.
+	* Valient Gough sent in patches to fixes multiple issues regarding building rpms.
+	* Fixed a few documentation issues. Several people, including Michael Rasmussen pointed these out.
+	* Fixed bug where files that can't be stat:ed cause fish to perform a double free when doing wildcard expantion.
+	* Fixed bug in manual page tab completion where completing a non-existing manual page results in bogus output.
+
+2005-05-18  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.9.1
+
+	* Fixed bug causing fish to hang on manual page tab completion if the whatis database can't be found. This bug has been reported by Gerben Versluis, Yongjian Xu and Ross Logan, but I couldn't pinpoint the source of the error before.
+	* Fixed bug causing fish to hang on command tab completion if the whatis database can't be found. This bug has been reported by Gerben Versluis, Yongjian Xu and Ross Logan, but I couldn't pinpoint the source of the error before.
+	* Fixed bug causing subshell output to be expanded
+	* Various minor bugs in gen_hdr2.c fixed, making documentation generation more robust. This change was sent in by Pierre Houston.
+	* The file gen_hdr.sh was simplified, and now works on OS X. This change was sent in by Pierre Houston.
+
+2005-05-14  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.9
+
+	* Changed quoting rules so that quoted and unquoted text can be mingled in one token
+	* Command names must may no longer contain variables. This is needed to correctly determine block scope.
+	* Functions definitions are now a block. The new syntax is much more suitable for complex functions.
+	* A new builtin, functions, for erasing and listing functions.
+	* Lots of minor cleanups, typedefed structs now end with '_t', a new string hashing function should decrease collisions in hash tables, etc.
+	* Variables can now be used as array indices.
+	* The set_color command can now be used to set the background color of the terminal.
+	* Added highlighting of grep matches and some additional colors for the ls command. These changes where submitted by Philip Ganchev.
+	* Minor performance increases.
+	* Moving a word left or right moves to the last space after the word.
+	* Completion pager is now much better at handling long completions and descriptions.
+	* Fish now uses the unicode ellipsis character instead of tripple dots when it can't fit a completion
+	* Line number for errors is now correctly calculated
+	* Error reports are more specific
+	* The commandline builtin can now be used to print the current commandline or append a string to the current commandline
+	* Fixed infinite loop when tab-completing inside curly brackets
+	* Fixed some of the IO redirection issues for functions
+	* Fixed bug that caused the $history variable to randomly misbehave.
+	* Fixed bug in file desctiption generation for strange filetypes.
+	* Fixed bug in completion description for variables where variable array descriptions could contain illegal characters
+
+2005-04-29  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.8
+
+	* Fixed bug that made 'make uninstall' miss some files 
+	* Fixed bug where subshells with quoted parentesis in them would result in errors
+	* Fixed crash bug when manually erasing a completion with 'complete -e'
+	* Fixed bug causing completion list to have larger column width than needed in some cases
+	* Fixed broken header causing wide characters to not work correctly. Both Adam Rice and Yongjian Xu have been a great help in making this work.
+	* Vastly improved startup time
+	* A large number of smaller updates and additions to the documentation
+	* Completion description for manual page completion is now the whatis information
+	* Completion description for commands now search manual sections 8, n, o and l in addition to section 1
+	* Matching parenthesis and quotes are now highlighted as they are typed, not just when moving the cursor over them
+	* Remove debug message on tab-completing directory from CDPATH
+	* Tab completion displays list of completions on single tap if completions have no common prefix
+	* Fish does not redraw the completion list on multiple tab taps
+	* It is now possible to specify an initial string for the read builtin
+	* New vared function for editing the value of a variable
+	* New commandline builtin for changing the command buffer
+	* History search now searches anywhere in the command
+	* History search highlights the matching string
+	* Changed output of set command when called with no arguments to be more readable
+
+2005-04-24  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.7.1
+
+	* Remove annoying debug message. Pointed out by Claudio Fontana.
+
+2005-04-23  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.7
+
+	* Support for environment variable arrays. $PATH[3] is the third element of the PATH variable, etc. 
+	* You can set individual variable elements using set, i.e. 'set PATH[2] foo' replaces the third element of PATH with 'foo'
+	* You can create an array using set, i.e. 'set foo bar baz' creates an array variable 'foo' with the elements 'bar' and 'baz'
+	* aliases have been replaced by functions. The Use 'help function' for an introduction to functions.
+	* Arguments to scripts/functions are now contained in the environment variable array $args. 
+	* functions can now be used in pipes
+	* complete.c: tab completion of directories in CDPATH for implicit cd now works.
+	* builtin.c: AThe read builtin now uses the internal line reading library, which means it supports all the advanced line reading capabilities that fish supports.
+	* : Debian packages are now available. Thank you to Dave Tait for making this happen. (This was really added to 1.6.1, but I forgot to mention it in the changelog)
+	* env.c: You can now access previous commands through the environment variable array named  history
+	* init/fish_function.fish: Added keyboard shortcut Control-r for replacing text in the current commandline
+	* input.c: Changed a few keyboard shortcuts
+	* doc_src/doc.hdr: Documentation updates
+	* : Fixed bug that broke several keyboard shortcuts
+	* reader.c: Check for double with characters through wcwidth
+
+2005-06-15  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.6.1
+
+	* util.c: Fixed buffer overflow with string_buffer
+
+	* complete.c: Fixed bug where tab-completion would not work on executable scripts
+
+	* reader.c: Fixed crash on terminals that do not provide some capabilities
+
+	* input.c: Alt-left and Alt-right keymappings where accidentaly reversed
+
+	
+2005-04-09  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.6
+
+	* init/fish_complete.fish: Updated completions for set_color. Thanks to Phil Ganchev for the bug report.
+
+	* reader.c: Added several new readline functions, suggested by Doug Dimick and Phil Ganchev.
+
+	* : ^C breaks all loops, conditional blocks, etc when waiting for a job to finish.
+
+	* builtin.c: Minor tweaks of completion syntax after suggestions from Tijmen Baarda.
+
+	* parser.c: If you just write in the name of a directory to fish, fish will implicitly assume this is a cd command and change to that directory. Thanks to Phil Ganchev for the suggestion.
+
+	* highlight.c: Highlighting of matching parentheses and quotes.
+
+	* doc_src/doc.hdr: The usual batch of documentation updates.
+
+	* parser.c: fish now has an upper limit on the number of recursive alias calls. This prevents trashing/crashing on bad aliases.
+
+	* parser.c: fish makes sure that an alias does not unconditionally call itself. This prevents the most trivial types of accidental infinite recursion.
+
+	
+2005-04-02  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.5.2
+
+	* : Fixed broken header that caused compilation issues
+
+
+2005-04-02  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.5.1
+
+	* reader.c: Fixed issue in completion display, where long completions might spill to the next line and cause uglyness
+
+	* highlight.c, reader.c: Fixed a bug in error explanation (^X) where gibberish would be printed in some cases
+
+	* reader.c: Fixed various minor issues with repainting the screen after status updates
+
+	* configure.ac, Makefile.in: fish can now be compiled without xsel by using './configure --without-xsel'. This means  fish can now be compiled on systems without X headers.
+
+	* input.c: Initial support for reading escape sequences for special keys from an inputrc file.
+	* input.c: Initial support for binding a command to a key. Meta-w runs whatis on the current command. Read the  /etc/fish_inputrc file for more information.
+
+	* reader.c: The LINES and COLUMNS variables are set to the correct terminal size
+
+
+2005-03-20  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.5
+
+	* complete.c: Tab completion of files now displays file descriptions now from the  mimetype database
+	* mimedb.c: New mimedb command, can be used to look up mimetype, description and default action for a file, as well as launch the default handler of a file.
+
+	* builtin.c: Jobs command prints CPU activity for jobs (Linux only)
+
+	* highlight.c: Added syntax highligting for subshells
+
+	* complete.c: Added tab completion inside of subshells
+
+	* reader.c: ^X prints comments on possible problems
+
+	* main.c: 'fish -h' starts the help browser
+
+	* : Non-builtin commands shipped with  fish have much improved help
+
+	* doc_src/doc.hdr: Various cleanups and rewrites of the documentation
+
+	* : Fixed accidental mixing of stream and fd based output which caused some status messages to become garbled.
+
+	* : Fixed bug hangup on commands such as 'ls --color|less', caused by process group weirdness. Thank you to Doug Dimick for reporting this issue.
+
+	* init/fish: Fixed problems with the linux frambuffer and unicode. Drew Ferguson reported this problem and went out of his way to help me solve it. Thanks!
+
+	* reader.c: Fixed Konsole Home and End keys not working. Thank you to Jason L. Buberel for reporting this issue.
+
+2005-03-11  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.4
+
+	* highlight.c: Unknown/misspelled options are now flaged red by syntax highlighting if tab completions have been specified for the 
+command.
+
+	* builtin.c: Added 'break' and 'continue' loop control builtins.
+	* builtin.c: Added 'switch' and 'case' builtins for conditionally executing 
+a block.
+
+	* doc_src/doc.hdr: Lots of smaller updates to the documentation.
+
+	* configure.ac: fish now uses autoconf to improve platform independance.
+
+	* kill.c: Fixed a bug in X clipboard support which could hang the shell. 
+
+	* expand.c: Fixed a bug where environment variables were escaped when they shouldn't be. This bug caused som tab-completions for the screen command to break in 1.3
+
+	* complete.c: Fixed bug causing problems for tab-completion if PATH elements end with a slash
+
+	* fish_tests.c: Took the first steps towards a test suite. So far only some simple syntax tests are performed.
+
+	* init/fish: If LANG is unspecified, a default value of en_US.UTF-8 is set
+
+	
+2005-03-07  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.3
+
+	* builtin.c: New syntax for if, while and for builtins. 
+	* : Added support for X copy and paste inside of  fish. Thank you to Stéphane Chazelas for pointers on how to do this.
+	* : Included the Xsel program, written by Conrad Parker, with  fish. This is the wonderful program that allows  fish to use X copy and paste.
+	* complete.c: Fixed bug causing slowdown in tab completion of wildcarded strings.
+	* reader.c: Avoid calling setupterm() more than once, since the NCurses version leaks memory and sometimes calls exit, possibly without an error message. 
+	* set_color.c: Fixed bug in the set_color command that could cause accidental underlining of text. Oh no!
+	* init/fish_complete.fish: Added tab completions for w, xsel, xprop, btdownloadcurses.py and btdownloadheadless.py commands.
+	* doc_src/coplete.txt: Rewrote documentation for the  complete command to make it easier to understand how command specific completions work.
+	* complete.c: Scripts with extensions are now described with language, i.e. Perl scripts are discribed as "Perl script", not "Executable"
+
+2005-02-27  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.2.1
+
+	* : Fixed obvious crash bug. I should REALLY try to put together a test suite for  fish.
+	* env.c: Added support for changing locale while  fish is running.
+
+2005-02-26  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.2
+
+	* INSTALL: The INSTALL file contained a dangerAous mistake, where users were advised to use a command which could cause an invalid /etc/shells. Thanks to Klaus Alexander Seistrup for spotting this.
+
+	* builtin.c: Fixed bug where sourcing a file when in interactive mode would reset signal handlers. 
+
+	* builtin_help.c: Builtin help pages are now stored as narrow character strings. This avoids escape sequences being improperly displayed on some platforms. It also decreases memory usage by a few kB.
+
+	* util.c: The comparison function used for sorting completions has been changed so it won't cause duplicate entries.
+
+	* wildcard.c: Avoid returning excessive number of completions when tab-completing a parameter ending with a wildcard. Thanks to Klaus Alexander Seistrup for suggesting this.
+
+	* doc_src/fish.1: Added a man-page pointing to the main  fish documentation. Thanks to Chris F.A. Johnson for suggesting this.
+
+	* doc_src/doc.hdr: Minor changes to the documentation.
+
+	* expand.c: Added tab completion for job id, with the job command as the description.
+
+	* init/fish_complete.fish: Added tab completions for ps, top, test, cvs, pine, chown and chgrp commands.
+
+	* builtin.c: Added CDPATH support for the cd builtin.
+
+	
+2005-02-16  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.1.1
+
+	* init/fish: Fixed typo that caused spurious error messages on startup
+
+	
+2005-02-16  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.1
+
+	* exec.c: Fixed possible segfault when using builtins in subshells (Subshells would sometimes read uninitialized data, no actual crashes encountered, but that might just be luck)
+
+	* expand.c: Completion descriptions for environment variables and for home directories (~USER).
+
+	* util.c: String sorting functions now sort strings that are a prefix to another string first (I.e. 'foo' comes before 'foobar') 
+
+	* reader.c: Tab completion pager colors can be set by user
+
+	* init/fish_complete.fish: Updated tab completion command description lookup to only use section 1 manuals
+
+	* doc_src/doc.hdr: Added documentation chapter on environment variables
+	* doc_src/doc.hdr: Added search targets for help sections 'color', 'prompt', 'title', 'variables' and 'expansion'
+	* doc_src/doc.hdr: Added BSD license info to documentation (About 100 lines of  fish code comes from OpenBSD and is BSD licensed)
+	* doc_src/doc.hdr: Minor additions and edits to documentation and initialization files
+
+	* init/fish_complete.fish: Added tab completions for bc, mv, uniq, wc and who
+
+	
+2005-02-13  Axel Liljencrantz  <axel@liljencrantz.se>
+
+	* Version 1.0
+
+	* Initial release, everything is new
+

Doxyfile.user

@@ -0,0 +1,161 @@
+PROJECT_NAME           = fish
+PROJECT_NUMBER         = 1
+OUTPUT_DIRECTORY       = user_doc
+CREATE_SUBDIRS         = NO
+OUTPUT_LANGUAGE        = English
+USE_WINDOWS_ENCODING   = NO
+BRIEF_MEMBER_DESC      = YES
+REPEAT_BRIEF           = YES
+ABBREVIATE_BRIEF       = YES
+ALWAYS_DETAILED_SEC    = NO
+INLINE_INHERITED_MEMB  = NO
+FULL_PATH_NAMES        = YES
+STRIP_FROM_PATH        = 
+STRIP_FROM_INC_PATH    = 
+SHORT_NAMES            = NO
+JAVADOC_AUTOBRIEF      = YES
+MULTILINE_CPP_IS_BRIEF = NO
+DETAILS_AT_TOP         = NO
+INHERIT_DOCS           = YES
+DISTRIBUTE_GROUP_DOC   = NO
+TAB_SIZE               = 8
+ALIASES                = 
+OPTIMIZE_OUTPUT_FOR_C  = YES
+OPTIMIZE_OUTPUT_JAVA   = NO
+SUBGROUPING            = YES
+EXTRACT_ALL            = NO
+EXTRACT_PRIVATE        = NO
+EXTRACT_STATIC         = YES
+EXTRACT_LOCAL_CLASSES  = YES
+EXTRACT_LOCAL_METHODS  = NO
+HIDE_UNDOC_MEMBERS     = NO
+HIDE_UNDOC_CLASSES     = NO
+HIDE_FRIEND_COMPOUNDS  = NO
+HIDE_IN_BODY_DOCS      = NO
+INTERNAL_DOCS          = NO
+CASE_SENSE_NAMES       = YES
+HIDE_SCOPE_NAMES       = NO
+SHOW_INCLUDE_FILES     = YES
+INLINE_INFO            = YES
+SORT_MEMBER_DOCS       = YES
+SORT_BRIEF_DOCS        = NO
+SORT_BY_SCOPE_NAME     = NO
+GENERATE_TODOLIST      = YES
+GENERATE_TESTLIST      = YES
+GENERATE_BUGLIST       = YES
+GENERATE_DEPRECATEDLIST= YES
+ENABLED_SECTIONS       = 
+MAX_INITIALIZER_LINES  = 30
+SHOW_USED_FILES        = YES
+SHOW_DIRECTORIES       = YES
+QUIET                  = NO
+WARNINGS               = YES
+WARN_IF_UNDOCUMENTED   = YES
+WARN_IF_DOC_ERROR      = YES
+WARN_FORMAT            = "$file:$line: $text"
+WARN_LOGFILE           = 
+INPUT                  = 
+FILE_PATTERNS          =  doc.h
+RECURSIVE              = NO
+EXCLUDE                = 
+EXCLUDE_SYMLINKS       = NO
+EXCLUDE_PATTERNS       = 
+EXAMPLE_PATH           = 
+EXAMPLE_PATTERNS       = 
+EXAMPLE_RECURSIVE      = NO
+IMAGE_PATH             = 
+INPUT_FILTER           = 
+FILTER_PATTERNS        = 
+FILTER_SOURCE_FILES    = NO
+SOURCE_BROWSER         = NO
+INLINE_SOURCES         = NO
+STRIP_CODE_COMMENTS    = YES
+REFERENCED_BY_RELATION = YES
+REFERENCES_RELATION    = YES
+VERBATIM_HEADERS       = YES
+ALPHABETICAL_INDEX     = NO
+COLS_IN_ALPHA_INDEX    = 5
+IGNORE_PREFIX          = 
+GENERATE_HTML          = YES
+HTML_OUTPUT            = html
+HTML_FILE_EXTENSION    = .html
+HTML_HEADER            = user_doc.head.html
+HTML_FOOTER            = 
+HTML_STYLESHEET        = 
+HTML_ALIGN_MEMBERS     = YES
+GENERATE_HTMLHELP      = NO
+CHM_FILE               = 
+HHC_LOCATION           = 
+GENERATE_CHI           = NO
+BINARY_TOC             = NO
+TOC_EXPAND             = NO
+DISABLE_INDEX          = YES
+ENUM_VALUES_PER_LINE   = 4
+GENERATE_TREEVIEW      = NO
+TREEVIEW_WIDTH         = 250
+GENERATE_LATEX         = YES
+LATEX_OUTPUT           = latex
+LATEX_CMD_NAME         = latex
+MAKEINDEX_CMD_NAME     = makeindex
+COMPACT_LATEX          = NO
+PAPER_TYPE             = a4wide
+EXTRA_PACKAGES         = 
+LATEX_HEADER           = 
+PDF_HYPERLINKS         = YES
+USE_PDFLATEX           = YES
+LATEX_BATCHMODE        = NO
+LATEX_HIDE_INDICES     = NO
+GENERATE_RTF           = NO
+RTF_OUTPUT             = rtf
+COMPACT_RTF            = NO
+RTF_HYPERLINKS         = NO
+RTF_STYLESHEET_FILE    = 
+RTF_EXTENSIONS_FILE    = 
+GENERATE_MAN           = NO
+MAN_OUTPUT             = man
+MAN_EXTENSION          = .3
+MAN_LINKS              = NO
+GENERATE_XML           = NO
+XML_OUTPUT             = xml
+XML_SCHEMA             = 
+XML_DTD                = 
+XML_PROGRAMLISTING     = YES
+GENERATE_AUTOGEN_DEF   = NO
+GENERATE_PERLMOD       = NO
+PERLMOD_LATEX          = NO
+PERLMOD_PRETTY         = YES
+PERLMOD_MAKEVAR_PREFIX = 
+ENABLE_PREPROCESSING   = YES
+MACRO_EXPANSION        = NO
+EXPAND_ONLY_PREDEF     = NO
+SEARCH_INCLUDES        = YES
+INCLUDE_PATH           = 
+INCLUDE_FILE_PATTERNS  = 
+PREDEFINED             = 
+EXPAND_AS_DEFINED      = 
+SKIP_FUNCTION_MACROS   = YES
+TAGFILES               = 
+GENERATE_TAGFILE       = 
+ALLEXTERNALS           = NO
+EXTERNAL_GROUPS        = YES
+PERL_PATH              = /usr/bin/perl
+CLASS_DIAGRAMS         = YES
+HIDE_UNDOC_RELATIONS   = YES
+HAVE_DOT               = NO
+CLASS_GRAPH            = NO
+COLLABORATION_GRAPH    = YES
+UML_LOOK               = NO
+TEMPLATE_RELATIONS     = NO
+INCLUDE_GRAPH          = NO
+INCLUDED_BY_GRAPH      = YES
+CALL_GRAPH             = YES
+GRAPHICAL_HIERARCHY    = YES
+DOT_IMAGE_FORMAT       = png
+DOT_PATH               = 
+DOTFILE_DIRS           = 
+MAX_DOT_GRAPH_WIDTH    = 750
+MAX_DOT_GRAPH_HEIGHT   = 1024
+MAX_DOT_GRAPH_DEPTH    = 0
+GENERATE_LEGEND        = YES
+DOT_CLEANUP            = YES
+SEARCHENGINE           = NO

INSTALL

@@ -0,0 +1,62 @@
+
+Known issues
+============
+
+Older versions of Doxygen has bugs in the man-page generation which
+cause the builtin help to render incorrectly. Version 1.2.14 is known
+to have this problem.
+
+In version 1.9.2, the installation prefix for fish rpms and debs has
+changed from /usr/local to /usr. The package should automatically
+change any instances of /usr/local/bin/fish in /etc/passwd to
+/usr/bin/fish, but some programs, like screen, may need to be
+restarted to notice the changes.  You may also run into problems when
+switching between using a package and personal builds.
+
+
+Prerequisites
+=============
+
+Fish requires the following packages to build:
+
+ - Doxygen
+ - Curses or Ncurses
+
+fish also relies on standard unix tools such as cat, cut, grep, sed,
+whoami and echo. Fish does not support cross-compilation, separate
+build directories or any other fancy configure options. Use a recent
+version of Doxygen, since older versions have bugs that make the
+builtin help pages render incorrectly. Version 1.2.14 is known to be
+broken.
+
+
+Simple install procedure
+========================
+
+ % ./configure
+ % make                                  #Compile fish
+ % make install                          #Install fish
+ % echo /usr/local/bin/fish >>/etc/shells #Add fish to list of shells
+
+If you wish to use fish as your default shell, use the following
+command:
+
+ % chsh -s /usr/local/bin/fish			 
+
+chsh will prompt you for your password, and change your default shell.
+
+
+Local install procedure
+=======================
+
+To install fish in your own home directory (typically as non-root),
+type:
+
+ % ./configure --prefix=$HOME
+ % make					# Compile fish
+ % make install			# Install fish
+
+You will not be able to use fish as the default shell unless you also
+add the corresponding line to /etc/shells, which kind of defeats the
+point of a local install. But you can at least build and run fish.
+

Makefile.in

@@ -0,0 +1,452 @@
+#
+# Copyright (C) 2005 Axel Liljencrantz
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+#
+
+#
+# Makefile for the fish shell. Can build fish and associated
+# applications, install them, recalculate dependencies and also create
+# binary distributions in tar.bz2 tar.gz and rpm formats.
+#
+
+#
+# The fish buildprocess is quite complex. Do not stare directly into
+# the Makefile. Doing so may cause nausea, dizziness and
+# hallucinations.
+#
+
+# Compiler flags
+
+CC := @CC@
+CFLAGS := @CFLAGS@ @INCLUDEDIR@ -Wall -std=gnu99 -fno-strict-aliasing
+CPPFLAGS=@CPPFLAGS@
+LDFLAGS:= -l@CURSESLIB@ @LDFLAGS@ @LIBDIR@
+INSTALL:=@INSTALL@
+
+prefix = @prefix@
+exec_prefix = @exec_prefix@
+bindir = @bindir@
+mandir = @mandir@
+sysconfdir = @sysconfdir@
+fishdir = @fishdir@
+fishfile = @fishfile@
+fishinputfile = @fishinputfile@
+docdir = @docdir@
+
+# All objects used by fish, that are compiled from an ordinary .c file
+# using an ordinary .h file.
+COMMON_OBJS := function.o builtin.o common.o complete.o env.o exec.o	\
+	expand.o highlight.o history.o kill.o parser.o proc.o reader.o		\
+	sanity.o tokenizer.o util.o wildcard.o wgetopt.o wutil.o input.o	\
+	output.o intern.o env_universal.o env_universal_common.o input_common.o
+
+# builtin_help.h exists, but builtin_help.c is autogenerated
+COMMON_OBJS_WITH_HEADER := builtin_help.o
+
+# main.c exists, but main.h does not, etc.
+COMMON_OBJS_WITH_CODE := builtin_set.o builtin_commandline.o 
+
+# All objects that the system needs to build fish
+FISH_OBJS := $(COMMON_OBJS) $(COMMON_OBJS_WITH_CODE) $(COMMON_OBJS_WITH_HEADER) main.o
+FISH_PAGER_OBJS := fish_pager.o common.o output.o util.o wutil.o tokenizer.o input_common.o env_universal.o env_universal_common.o
+FISH_TESTS_OBJS := $(COMMON_OBJS) $(COMMON_OBJS_WITH_CODE) $(COMMON_OBJS_WITH_HEADER) fish_tests.o 
+FISHD_OBJS := fishd.o env_universal_common.o common.o util.o wutil.o	\
+
+
+#All objects that the system needs to build mimedb
+MIME_OBJS := mimedb.o xdgmimealias.o xdgmime.o xdgmimeglob.o	\
+	xdgmimeint.o xdgmimemagic.o xdgmimeparent.o
+
+#
+# Files containing documentation for builtins. Should be listed
+# alphabetically, since this is the order in which they will be written
+# in the help file.
+#
+
+BUILTIN_DOC_SRC := doc_src/source.txt doc_src/and.txt					\
+	doc_src/begin.txt doc_src/bg.txt doc_src/bind.txt					\
+	doc_src/break.txt doc_src/builtin.txt doc_src/case.txt				\
+	doc_src/cd.txt doc_src/command.txt doc_src/commandline.txt			\
+	doc_src/complete.txt doc_src/continue.txt doc_src/else.txt			\
+	doc_src/end.txt doc_src/eval.txt doc_src/exec.txt doc_src/exit.txt	\
+	doc_src/fg.txt doc_src/for.txt doc_src/function.txt					\
+	doc_src/functions.txt doc_src/if.txt doc_src/jobs.txt				\
+	doc_src/not.txt doc_src/or.txt doc_src/random.txt					\
+	doc_src/return.txt doc_src/read.txt doc_src/set.txt					\
+	doc_src/switch.txt doc_src/while.txt
+
+#
+# Files generated by running doxygen on the files in $(BUILTIN_DOC_SRC)
+#
+BUILTIN_DOC_HDR := $(BUILTIN_DOC_SRC:.txt=.doxygen)
+
+#
+# Files containing documentation for external commands. Should be listed
+# alphabetically, since this is the order in which they will be written
+# in the help file.
+#
+
+CMD_DOC_SRC := doc_src/count.txt doc_src/dirh.txt doc_src/dirs.txt	\
+	doc_src/help.txt doc_src/mimedb.txt doc_src/nextd.txt			\
+	doc_src/open.txt doc_src/popd.txt doc_src/prevd.txt				\
+	doc_src/pushd.txt doc_src/set_color.txt doc_src/tokenize.txt
+
+#
+# Files generated by running doxygen on the files in $(CMD_DOC_SRC)
+#
+CMD_DOC_HDR := $(CMD_DOC_SRC:.txt=.doxygen)
+
+TEST_IN := $(wildcard tests/test*.in)
+
+#
+# Files that should be added to the tar archives
+#
+
+# Files in ./doc_src/
+DOC_SRC_DIR_FILES := doc_src/Doxyfile.in doc_src/doc.hdr	\
+	$(BUILTIN_DOC_SRC) $(CMD_DOC_SRC) doc_src/fish.1.in
+
+# Files in ./
+MAIN_DIR_FILES := Doxyfile Doxyfile.user Makefile.in configure			\
+    configure.ac config.h.in install-sh set_color.c count.c				\
+    key_reader.c tokenize.c gen_hdr.sh gen_hdr2.c $(MIME_OBJS:.o=.h)	\
+    $(MIME_OBJS:.o=.c) $(COMMON_OBJS_WITH_HEADER:.o=.h)					\
+    $(COMMON_OBJS:.o=.h) $(COMMON_OBJS_WITH_CODE:.o=.c)					\
+    $(COMMON_OBJS:.o=.c) builtin_help.hdr fish.spec.in INSTALL README	\
+    user_doc.head.html xsel-0.9.6.tar ChangeLog config.sub				\
+    config.guess fish_tests.c main.c fish_pager.c fishd.c
+
+# Files in ./init/
+INIT_DIR_FILES :=init/fish.in init/fish_complete.fish	\
+	init/fish_function.fish init/fish_inputrc			\
+	init/fish_interactive.fish
+
+# Files in ./tests/
+TESTS_DIR_FILES := $(TEST_IN) $(TEST_IN:.in=.out) $(TEST_IN:.in=.err)	\
+	$(TEST_IN:.in=.status) tests/test.fish tests/gen_output.fish
+
+COMPLETIONS_DIR_FILES := $(wildcard init/completions/*.fish)
+
+# Programs to build
+PROGRAMS:=fish set_color tokenize @XSEL@ mimedb count fish_pager fishd
+
+# Manuals to install
+MANUALS:=doc_src/fish.1 @XSEL_MAN_PATH@			\
+	doc_src/builtin_doc/man/man1/mimedb.1		\
+	doc_src/builtin_doc/man/man1/set_color.1	\
+	doc_src/builtin_doc/man/man1/tokenize.1		\
+	doc_src/builtin_doc/man/man1/count.1
+
+#Make everything needed for installing fish
+all: $(PROGRAMS) user_doc 
+
+# User documentation, describing the features of the fish shell.
+user_doc: doc.h Doxyfile.user user_doc.head.html
+	doxygen Doxyfile.user
+
+#Source code documentation. Also includes user documentation.
+doc: *.h *.c doc.h Doxyfile builtin_help.c
+	doxygen; 
+
+# PDF version of the source code documentation.
+doc/refman.pdf: doc 
+	cd doc/latex; 
+	make; 
+	mv refman.pdf ..; 
+	cd ../..; 
+	rm -r doc/latex;
+
+test: $(PROGRAMS) fish_tests
+	./fish_tests; cd tests; ../fish <test.fish;
+
+xsel-0.9.6:
+	tar -xf xsel-0.9.6.tar
+
+xsel-0.9.6/xsel: xsel-0.9.6
+	cd xsel-0.9.6; ./configure; make
+
+# doc.h is a compilation of the various snipptes of text used both for
+# the user documentation and for internal help functions into a single
+# file that can be parsed dy Doxygen to generate the user
+# documentation.
+doc.h:$(BUILTIN_DOC_SRC) $(CMD_DOC_SRC) doc_src/doc.hdr
+	cat doc_src/doc.hdr >doc.h; 
+	echo "/** \page builtins Builtin commands" >>doc.h; 
+	cat $(BUILTIN_DOC_SRC) >>doc.h; 
+	echo "*/" >>doc.h
+	echo "/** \page commands External commands" >>doc.h; 
+	echo "\c fish is shipped with commands which do not use any internal parts of the shell, and are therefore not written as builtins, but separate commands." >>doc.h
+	cat $(CMD_DOC_SRC) >>doc.h; 
+	echo "*/" >>doc.h
+
+# This rule creates complete doxygen headers from each of the various
+# snipptes of text used both for the user documentation and for
+# internal help functions, that can be parsed to Doxygen to generate
+# the internal help function text.
+%.doxygen:%.txt
+	echo  "/** \page " `basename $*` >$@; 
+	cat $*.txt >>$@; 
+	echo "*/" >>$@
+
+# Generate the internal help functions by making doxygen create
+# man-pages which are then converted into C code. The convertion path
+# looks like this: 
+#
+#   .txt file
+#       ||
+#     (make)
+#       ||
+#       \/
+#  .doxygen file
+#       ||
+#    (doxygen)
+#       ||
+#       \/
+#    man file
+#       ||
+#      (man)
+#       ||
+#       \/
+#   formated text 
+#   with escape 
+#    sequences
+#       ||
+#       \/
+#   (gen_hdr2)
+#       ||
+#       \/
+#    .c file
+#
+# Which is an awful, clunky and ugly way of producing
+# documentation. There ought to be something simpler.
+
+builtin_help.c: $(BUILTIN_DOC_HDR) doc_src/count.doxygen gen_hdr2 gen_hdr.sh builtin_help.hdr $(CMD_DOC_HDR)
+	cd doc_src; doxygen; cd ..; 
+	cp builtin_help.hdr builtin_help.c; 
+	for i in $(BUILTIN_DOC_HDR) doc_src/count.doxygen ; do \
+		echo '    hash_put( &tbl, L"'`basename $$i .doxygen`'",' >>$@; \
+		./gen_hdr.sh $$i >>$@; \
+		echo " );" >>$@; \
+		echo >>$@; \
+	done;
+	echo "}" >>builtin_help.c
+
+#
+# Generate help texts for external fish commands, like set_color and
+# mimedb. Depends on builtin_help.c to make sure doxygen gets run to
+# generate the man files.
+#
+%.c : %.doxygen gen_hdr2 builtin_help.c
+	echo "// This file was automatically generated, do not edit" >$@
+	echo "#include <stdlib.h>" >>$@
+	echo "#include <stdio.h>" >>$@
+	echo >>$@
+	echo "void print_help()" >>$@
+	echo "{" >>$@
+	echo '    printf( "%s",' >>$@
+	./gen_hdr.sh $*.doxygen >>$@
+	echo ");" >>$@
+	echo "}" >>$@
+#man -- doc_src/builtin_doc/man/man1/`basename $@ .c`.1 | cat -s | ./gen_hdr2 >>$@
+
+install: all 
+	$(INSTALL) -m 755 -d $(DESTDIR)$(bindir) 
+	for i in $(PROGRAMS); do\
+		$(INSTALL) -m 755 $$i $(DESTDIR)$(bindir) ; \
+	done;
+	$(INSTALL) -m 755 -d $(DESTDIR)$(sysconfdir)$(fishdir)
+	$(INSTALL) -m 755 -d $(DESTDIR)$(sysconfdir)$(fishdir)/completions
+	$(INSTALL) -m 644 init/fish $(DESTDIR)$(sysconfdir)$(fishfile)
+	for i in init/fish_interactive.fish init/fish_function.fish init/fish_complete.fish ; do \
+		$(INSTALL) -m 644 $$i $(DESTDIR)$(sysconfdir)$(fishdir); \
+	done;
+	for i in $(COMPLETIONS_DIR_FILES); do \
+		$(INSTALL) -m 644 $$i $(DESTDIR)$(sysconfdir)$(fishdir)/completions/; \
+	done;
+	$(INSTALL) -m 644 init/fish_inputrc $(DESTDIR)$(sysconfdir)$(fishinputfile);
+	$(INSTALL) -m 755 -d $(DESTDIR)$(docdir)
+	for i in user_doc/html/* ChangeLog; do \
+		$(INSTALL) -m 644 $$i $(DESTDIR)$(docdir); \
+	done;
+	$(INSTALL) -m 755 -d $(DESTDIR)$(mandir)/man1
+	for i in $(MANUALS); do \
+		$(INSTALL) -m 644 $$i $(DESTDIR)$(mandir)/man1/; \
+	done;
+	@echo If you want to use fish as the default shell, remember to first
+	@echo add the line \'$(DESTDIR)$(bindir)/fish\' to the file \'/etc/shells\'.
+
+uninstall:
+	for i in $(PROGRAMS); do \
+		rm -f $(DESTDIR)$(bindir)/$$i; \
+	done;
+	rm -f $(DESTDIR)$(bindir)/xsel
+	rm -f $(DESTDIR)$(sysconfdir)$(fishfile)
+	rm -f $(DESTDIR)$(sysconfdir)$(fishinputfile)
+	rm -r $(DESTDIR)$(sysconfdir)$(fishdir)
+	rm -r $(DESTDIR)$(docdir)
+	for i in fish.1* @XSEL_MAN@ mimedb.1* set_color.1* tokenize.1* count.1*; do \
+		rm $(DESTDIR)$(mandir)/man1/$$i; \
+	done;
+
+# The fish shell
+fish: $(FISH_OBJS) 
+	$(CC) $(FISH_OBJS) $(LDFLAGS) -o $@
+
+fish_pager: $(FISH_PAGER_OBJS)
+	$(CC) $(FISH_PAGER_OBJS) $(LDFLAGS) -o $@
+
+fishd: $(FISHD_OBJS)
+	$(CC) $(FISHD_OBJS) $(LDFLAGS) -o $@
+
+fish_tests: $(FISH_TESTS_OBJS)
+	$(CC) $(FISH_TESTS_OBJS) $(LDFLAGS) -o $@
+
+
+mimedb: $(MIME_OBJS) util.o common.o doc_src/mimedb.c
+	$(CC) ${MIME_OBJS} util.o common.o doc_src/mimedb.c $(LDFLAGS) -o $@
+
+set_color: set_color.o doc_src/set_color.c 
+	$(CC) set_color.o doc_src/set_color.c $(LDFLAGS) -o $@
+
+tokenize: tokenize.o doc_src/tokenize.c 
+	$(CC) tokenize.o doc_src/tokenize.c $(LDFLAGS) -o $@
+
+# Test program for the tokenizer library
+tokenizer_test: tokenizer.c tokenizer.h util.o wutil.o common.o
+	$(CC) ${CFLAGS} tokenizer.c util.o wutil.o common.o -D TOKENIZER_TEST $(LDFLAGS) -o $@
+
+depend:
+	makedepend -fMakefile.in -Y *.c 
+
+# Copy all the source files into a new directory and use tar to create
+# an archive from it. Simplest way I could think of to make an archive
+# witout backups, autogenerated files, etc.
+#
+# Uses install instead of mkdir so build won't fail if the directory 
+# exists
+fish-@PACKAGE_VERSION@.tar: $(DOC_SRC_DIR_FILES) $(MAIN_DIR_FILES) $(INIT_DIR_FILES) $(TEST_DIR_FILES) $(COMPLETIONS_DIR_FILES) ChangeLog
+	rm -rf fish-@PACKAGE_VERSION@
+	$(INSTALL) -d fish-@PACKAGE_VERSION@
+	$(INSTALL) -d fish-@PACKAGE_VERSION@/doc_src
+	$(INSTALL) -d fish-@PACKAGE_VERSION@/init
+	$(INSTALL) -d fish-@PACKAGE_VERSION@/init/completions
+	$(INSTALL) -d fish-@PACKAGE_VERSION@/tests
+	cp -f $(DOC_SRC_DIR_FILES) fish-@PACKAGE_VERSION@/doc_src
+	cp -f $(MAIN_DIR_FILES) fish-@PACKAGE_VERSION@/
+	cp -f $(INIT_DIR_FILES) fish-@PACKAGE_VERSION@/init/
+	cp -f $(COMPLETIONS_DIR_FILES) fish-@PACKAGE_VERSION@/init/completions/
+	cp -f $(TESTS_DIR_FILES) fish-@PACKAGE_VERSION@/tests/
+	tar -c fish-@PACKAGE_VERSION@ >fish-@PACKAGE_VERSION@.tar
+	rm -rf fish-@PACKAGE_VERSION@
+
+fish-@PACKAGE_VERSION@.tar.gz: fish-@PACKAGE_VERSION@.tar
+	gzip -f --best -c fish-@PACKAGE_VERSION@.tar >fish-@PACKAGE_VERSION@.tar.gz
+
+fish-@PACKAGE_VERSION@.tar.bz2: fish-@PACKAGE_VERSION@.tar
+	bzip2 -f --best -k fish-@PACKAGE_VERSION@.tar
+
+# Create .rpm file for the current systems architecture and an
+# .src.rpm file.
+rpm: fish-@PACKAGE_VERSION@.tar.bz2
+	cp fish.spec /usr/src/redhat/SPECS/
+	cp fish-@PACKAGE_VERSION@.tar.bz2 /usr/src/redhat/SOURCES/
+	rpmbuild -ba --clean /usr/src/redhat/SPECS/fish.spec
+	mv /usr/src/redhat/RPMS/*/fish*@PACKAGE_VERSION@*.rpm .
+	mv /usr/src/redhat/SRPMS/fish*@PACKAGE_VERSION@*.src.rpm .
+
+clean:
+	rm -f *.o doc.h doc_src/*.doxygen doc_src/*.c builtin_help.c 
+	rm -f config.status config.log config.h Makefile
+	rm -f tokenizer_test fish key_reader set_color tokenize gen_hdr2 mimedb
+	rm -f fish-@PACKAGE_VERSION@.tar 
+	rm -f fish-@PACKAGE_VERSION@.tar.gz 
+	rm -f fish-@PACKAGE_VERSION@.tar.bz2
+	rm -rf doc; 
+	rm -rf user_doc; 
+	rm -rf doc_src/builtin_doc
+	rm -rf fish-@PACKAGE_VERSION@
+	rm -rf xsel-0.9.6/
+
+# DO NOT DELETE THIS LINE -- make depend depends on it.
+
+builtin.o: config.h util.h wutil.h builtin.h function.h complete.h proc.h
+builtin.o: parser.h reader.h env.h expand.h common.h wgetopt.h sanity.h
+builtin.o: tokenizer.h builtin_help.h wildcard.h input_common.h input.h
+builtin.o: intern.h
+builtin_commandline.o: config.h util.h builtin.h common.h wgetopt.h reader.h
+builtin_commandline.o: proc.h parser.h tokenizer.h input_common.h input.h
+builtin_help.o: config.h util.h common.h builtin_help.h
+builtin_set.o: config.h util.h builtin.h env.h expand.h common.h wgetopt.h
+builtin_set.o: proc.h parser.h
+common.o: config.h util.h wutil.h common.h expand.h proc.h wildcard.h
+common.o: parser.h
+complete.o: config.h util.h tokenizer.h wildcard.h proc.h parser.h function.h
+complete.o: complete.h builtin.h env.h exec.h expand.h common.h reader.h
+complete.o: history.h intern.h wutil.h
+env.o: config.h util.h wutil.h proc.h common.h env.h sanity.h expand.h
+env.o: history.h reader.h parser.h env_universal.h env_universal_common.h
+env_universal.o: util.h common.h wutil.h env_universal_common.h
+env_universal_common.o: util.h common.h wutil.h env_universal_common.h
+exec.o: config.h util.h common.h wutil.h proc.h exec.h parser.h builtin.h
+exec.o: function.h env.h wildcard.h sanity.h expand.h env_universal.h
+exec.o: env_universal_common.h
+expand.o: config.h util.h common.h wutil.h env.h proc.h parser.h expand.h
+expand.o: wildcard.h exec.h tokenizer.h complete.h
+fishd.o: util.h common.h wutil.h env_universal_common.h
+fish_pager.o: config.h util.h wutil.h common.h complete.h output.h
+fish_pager.o: input_common.h env_universal.h env_universal_common.h
+fish_tests.o: config.h util.h common.h proc.h reader.h builtin.h function.h
+fish_tests.o: complete.h wutil.h env.h expand.h parser.h tokenizer.h
+function.o: config.h util.h function.h proc.h parser.h common.h intern.h
+highlight.o: config.h util.h wutil.h highlight.h tokenizer.h proc.h parser.h
+highlight.o: builtin.h function.h env.h expand.h sanity.h common.h complete.h
+highlight.o: output.h
+history.o: config.h util.h wutil.h history.h common.h reader.h env.h sanity.h
+input.o: config.h util.h wutil.h reader.h proc.h common.h sanity.h
+input.o: input_common.h input.h parser.h env.h expand.h
+input_common.o: config.h util.h common.h wutil.h input_common.h
+input_common.o: env_universal.h env_universal_common.h
+intern.o: config.h util.h common.h intern.h
+kill.o: config.h util.h wutil.h kill.h proc.h sanity.h common.h env.h
+kill.o: expand.h exec.h parser.h
+main.o: config.h util.h common.h reader.h builtin.h function.h complete.h
+main.o: wutil.h env.h sanity.h proc.h parser.h expand.h intern.h
+mimedb.o: config.h xdgmime.h util.h
+output.o: config.h util.h wutil.h expand.h common.h output.h highlight.h
+parser.o: config.h util.h common.h wutil.h proc.h parser.h tokenizer.h exec.h
+parser.o: wildcard.h function.h builtin.h builtin_help.h env.h expand.h
+parser.o: reader.h sanity.h
+proc.o: config.h util.h wutil.h proc.h common.h reader.h sanity.h env.h
+reader.o: config.h util.h wutil.h highlight.h reader.h proc.h parser.h
+reader.o: complete.h history.h common.h sanity.h env.h exec.h expand.h
+reader.o: tokenizer.h kill.h input_common.h input.h function.h output.h
+sanity.o: config.h util.h common.h sanity.h proc.h history.h reader.h kill.h
+sanity.o: wutil.h
+set_color.o: config.h
+tokenize.o: config.h
+tokenizer.o: config.h util.h wutil.h tokenizer.h common.h wildcard.h
+util.o: config.h util.h common.h wutil.h
+wgetopt.o: config.h wgetopt.h wutil.h
+wildcard.o: config.h util.h wutil.h complete.h common.h wildcard.h reader.h
+wildcard.o: expand.h
+wutil.o: config.h util.h common.h wutil.h
+xdgmimealias.o: xdgmimealias.h xdgmime.h xdgmimeint.h
+xdgmime.o: xdgmime.h xdgmimeint.h xdgmimeglob.h xdgmimemagic.h xdgmimealias.h
+xdgmime.o: xdgmimeparent.h
+xdgmimeglob.o: xdgmimeglob.h xdgmime.h xdgmimeint.h
+xdgmimeint.o: xdgmimeint.h xdgmime.h
+xdgmimemagic.o: xdgmimemagic.h xdgmime.h xdgmimeint.h
+xdgmimeparent.o: xdgmimeparent.h xdgmime.h xdgmimeint.h

README

@@ -0,0 +1,15 @@
+How to find documentation for fish
+==================================
+
+The fish documentation is distributed in an intermediate format. To
+view it, you have to type:
+
+  % make user_doc
+
+Which will create the directory user_doc, containing html
+documentation for fish. If you build and install fish, the
+documentation will be available through the 'help' builtin.
+
+After installation, you can start fish by typing fish in the
+terminal. After fish has started, try using the help command for more
+information.

builtin.h

@@ -0,0 +1,101 @@
+/** \file builtin.h
+	Prototypes for functions for executing builtin functions.
+*/
+
+
+enum
+{
+	COMMAND_NOT_BUILTIN,
+	BUILTIN_REGULAR,
+	BUILTIN_FUNCTION
+}
+;
+
+#define BUILTIN_ERR_MISSING L": Expected argument"
+#define BUILTIN_ERR_COMBO L": Invalid combination of options"
+#define BUILTIN_ERR_GLOCAL L": Variable can only be one of universal, global and local"
+#define BUILTIN_ERR_EXPUNEXP L": Variable can't be both exported and unexported"
+#define BUILTIN_ERR_UNKNOWN	L": Unknown option"
+
+/**
+   Stringbuffer used to represent standard output
+*/
+extern string_buffer_t *sb_out;
+
+/**
+   Stringbuffer used to represent standard error
+*/
+extern string_buffer_t *sb_err;
+
+/**
+   Kludge. Tells builtins if output is to screen
+*/
+extern int builtin_out_redirect;
+
+/**
+   Kludge. Tells builtins if error is to screen
+*/
+extern int builtin_err_redirect;
+
+
+/**
+   Initialize builtin data. 
+*/
+void builtin_init();
+
+/**
+   Destroy builtin data.
+*/
+void builtin_destroy();
+
+/**
+  Is there a builtin command with the given name?
+*/
+int builtin_exists( wchar_t *cmd );
+
+/**
+  Execute a builtin command 
+
+  \param argv Array containing the command and parameters 
+  of the builtin.  The list is terminated by a
+  null pointer. This syntax resembles the syntax 
+  for exec.
+
+  \return the exit status of the builtin command
+*/
+int builtin_run( wchar_t **argv );
+
+/**
+  Insert all builtin names into l. These are not copies of the strings and should not be freed after use.
+*/
+void builtin_get_names( array_list_t *list );
+
+/**
+   Pushes a new set of input/output to the stack. The new stdin is supplied, a new set of output string_buffer_ts is created.
+*/
+void builtin_push_io( int stdin_fd );
+
+/**
+   Pops a set of input/output from the stack. The output string_buffer_ts are destroued, but the input file is not closed.
+*/
+void builtin_pop_io();
+
+
+/**
+   Return a one-line description of the specified builtin
+*/
+const wchar_t *builtin_get_desc( const wchar_t *b );
+
+/**
+   Counts the number of non null pointers in the specified array
+*/
+int builtin_count_args( wchar_t **argv );
+
+/**
+   Print help for the specified builtin. If \c b is sb_err, also print the line information
+*/
+void builtin_print_help( wchar_t *cmd, string_buffer_t *b );
+
+
+int builtin_set(wchar_t **argv);
+int builtin_commandline(wchar_t **argv);

builtin_commandline.c

@@ -0,0 +1,461 @@
+/** \file builtin_commandline.c Functions defining the commandline builtin
+
+Functions used for implementing the commandline builtin. 
+
+*/
+#include <stdlib.h>
+#include <stdio.h>
+#include <wchar.h>
+#include <wctype.h>
+#include <sys/types.h>
+#include <termios.h>
+#include <signal.h>
+
+#include "config.h"
+#include "util.h"
+#include "builtin.h"
+#include "common.h"
+#include "wgetopt.h"
+#include "reader.h"
+#include "proc.h"
+#include "parser.h"
+#include "tokenizer.h"
+#include "input_common.h"
+#include "input.h"
+
+/**
+   Which part of the comandbuffer are we operating on
+*/
+enum
+{
+	STRING_MODE=1, // Operate on entire buffer
+	JOB_MODE, // Operate on job under cursor
+	PROCESS_MODE, // Operate on process under cursor
+	TOKEN_MODE // Operate on token under cursor
+}
+	;
+
+/**
+   For text insertion, how should it be done
+*/
+enum
+{
+	REPLACE_MODE=1, // Replace current text
+	INSERT_MODE, // Insert at cursor position
+	APPEND_MODE // Insert at end of current token/command/buffer
+}
+	;
+
+
+static void replace_part( wchar_t *begin,
+						  wchar_t *end,
+						  wchar_t *insert,
+						  int append_mode )
+{
+	wchar_t *buff = reader_get_buffer();
+	string_buffer_t out;
+	int out_pos=reader_get_cursor_pos();
+					
+	sb_init( &out );
+
+//					wchar_t *tmp = wcsndup( begin, end-begin );
+					
+//					fwprintf( stderr, L"Commandline '%ls', current command '%ls'\n", reader_get_buffer(), tmp );
+					
+	sb_append_substring( &out, buff, begin-buff );
+					
+	switch( append_mode)
+	{
+		case REPLACE_MODE:
+		{
+							
+			sb_append( &out, insert );
+			out_pos = wcslen( insert ) + (begin-buff);							
+			break;
+							
+		}
+		case APPEND_MODE:
+		{
+			sb_append_substring( &out, begin, end-begin );
+			sb_append( &out, insert );
+			break;
+		}
+		case INSERT_MODE:
+		{
+			int cursor = reader_get_cursor_pos() -(begin-buff);
+			sb_append_substring( &out, begin, cursor );
+			sb_append( &out, insert );
+			sb_append_substring( &out, begin+cursor, end-begin-cursor );
+			out_pos +=  wcslen( insert );							
+			break;
+		}
+	}
+	sb_append( &out, end );
+	reader_set_buffer( (wchar_t *)out.buff, out_pos );
+	sb_destroy( &out );
+}
+	
+void write_part( wchar_t *begin, 
+				 wchar_t *end, 
+				 int cut_at_cursor, 
+				 int tokenize )
+{	
+	tokenizer tok;
+	string_buffer_t out;
+	wchar_t *buff;
+	int pos;
+
+	pos = reader_get_cursor_pos()-(begin-reader_get_buffer());
+
+	if( tokenize )
+	{
+					
+		buff = wcsndup( begin, end-begin );
+//		fwprintf( stderr, L"Subshell: %ls, end char %lc\n", buff, *end );
+		sb_init( &out );
+					
+		for( tok_init( &tok, buff, TOK_ACCEPT_UNFINISHED );
+			 tok_has_next( &tok );
+			 tok_next( &tok ) )
+		{
+			if( (cut_at_cursor) &&
+				(tok_get_pos( &tok)+wcslen(tok_last( &tok)) >= pos) )
+				break;
+		
+//			fwprintf( stderr, L"Next token %ls\n", tok_last( &tok ) );
+
+			switch( tok_last_type( &tok ) )
+			{
+				case TOK_STRING:
+					sb_append2( &out, tok_last( &tok), L"\n", 0 );
+					break;
+								
+			}						
+		}
+					
+		if( out.buff )
+			sb_append( sb_out,
+					   (wchar_t *)out.buff );
+					
+		free( buff );
+		tok_destroy( &tok );
+		sb_destroy( &out );
+	}
+	else
+	{
+		if( cut_at_cursor )
+		{
+			end = begin+pos;			
+		}
+		sb_append_substring( sb_out, begin, end-begin );		
+		sb_append( sb_out, L"\n" );		
+	}
+}
+
+
+/**
+   The commandline builtin. It is used for specifying a new value for
+   the commandline.
+*/
+int builtin_commandline( wchar_t **argv )
+{
+
+	int buffer_part=0;
+	int cut_at_cursor=0;
+	
+	int argc = builtin_count_args( argv );
+	int append_mode=0;
+
+	int function_mode = 0;
+	
+	int tokenize = 0;	
+
+	if( !reader_get_buffer() )
+	{
+		sb_append2( sb_err,
+					argv[0],
+					L": Can not set commandline in non-interactive mode\n",
+					0 );
+		builtin_print_help( argv[0], sb_err );
+		return 1;		
+	}	
+
+	woptind=0;
+
+	while( 1 )
+	{
+		const static struct woption
+			long_options[] =
+			{
+				{
+					L"append", no_argument, 0, 'a'
+				}
+				,
+				{
+					L"insert", no_argument, 0, 'i'
+				}
+				,
+				{
+					L"replace", no_argument, 0, 'r'
+				}
+				,
+				{
+					L"current-job", no_argument, 0, 'j'
+				}
+				,
+				{
+					L"current-process", no_argument, 0, 'p'
+				}
+				,
+				{
+					L"current-token", no_argument, 0, 't'
+				}
+				,		
+				{
+					L"current-buffer", no_argument, 0, 'b'
+				}
+				,		
+				{
+					L"cut-at-cursor", no_argument, 0, 'c'
+				}
+				,
+				{
+					L"function", no_argument, 0, 'f'
+				}
+				,
+				{
+					L"tokenize", no_argument, 0, 'o'
+				}
+				,
+				{
+					0, 0, 0, 0 
+				}
+			}
+		;		
+		
+		int opt_index = 0;
+		
+		int opt = wgetopt_long( argc,
+								argv, 
+								L"aijpctwfo", 
+								long_options, 
+								&opt_index );
+		if( opt == -1 )
+			break;
+			
+		switch( opt )
+		{
+			case 0:
+				if(long_options[opt_index].flag != 0)
+					break;
+				sb_append2( sb_err, 
+							argv[0],
+							BUILTIN_ERR_UNKNOWN,
+							L" ",
+							long_options[opt_index].name,
+							L"\n",
+							0 );
+				builtin_print_help( argv[0], sb_err );
+
+				return 1;
+				
+			case L'a':
+				append_mode = APPEND_MODE;
+				break;
+				
+			case L'i':
+				append_mode = INSERT_MODE;
+				break;
+				
+			case 'c':
+				cut_at_cursor=1;
+				break;
+				
+			case 't':
+				buffer_part = TOKEN_MODE;
+				break;
+
+			case 'j':
+				buffer_part = JOB_MODE;
+				break;
+
+			case 'p':
+				buffer_part = PROCESS_MODE;
+				break;
+
+			case 'f':
+				function_mode=1;
+				break;
+
+			case 'o':
+				tokenize=1;
+				break;
+
+			case L'?':
+				builtin_print_help( argv[0], sb_err );				
+				return 1;	
+		}
+	}		
+
+	if( function_mode )
+	{
+		int i;
+		
+		/*
+		  Check for invalid switch combinations
+		*/
+		if( buffer_part || cut_at_cursor || append_mode || tokenize )
+		{
+			sb_append2(sb_err,
+					   argv[0],
+					   BUILTIN_ERR_COMBO,
+					   L"\n",
+					   parser_current_line(),
+					   L"\n",
+					   0);
+			return 1;
+		}
+		
+
+		if( argc == woptind )
+		{
+			sb_append2( sb_err,
+						argv[0],
+						BUILTIN_ERR_MISSING,
+						L"\n",
+						parser_current_line(),
+						L"\n",
+						0 );
+			builtin_print_help( argv[0], sb_err );
+			return 1;
+ 		}
+		for( i=woptind; i<argc; i++ )
+		{
+			wint_t c = input_get_code( argv[i] );
+			if( c != -1 )
+			{
+//				fwprintf( stderr, L"Add function %ls (%d)\n", argv[i], c );
+				/*
+				  input_unreadch inserts the specified keypress or
+				  readline function at the top of the stack of unused
+				  keypresses
+				*/
+				input_unreadch(c);
+			}
+			else
+			{
+//				fwprintf( stderr, L"BLUR %ls %d\n", argv[i], c );
+				sb_append2( sb_err,
+							argv[0],
+							L": Unknown readline function '",
+							argv[i],
+							L"'\n",
+							parser_current_line(),
+							L"\n",
+							0 );
+				builtin_print_help( argv[0], sb_err );
+				return 1;
+			}
+		}
+		
+		return 0;		
+	}
+	
+	/*
+	  Check for invalid switch combinations
+	*/
+	if( argc-woptind > 1 )
+	{
+		
+		sb_append2( sb_err,
+					argv[0],
+					L": Too many arguments\n",
+					0 );
+		builtin_print_help( argv[0], sb_err );
+		return 1;
+	}
+
+	if( (tokenize || cut_at_cursor) && (argc-woptind) )
+	{
+		
+		sb_append2( sb_err,
+					argv[0],
+					BUILTIN_ERR_COMBO,
+					L",\n --cut-at-cursor and --tokenize can not be used when setting the commandline",
+					0 );
+		builtin_print_help( argv[0], sb_err );
+		return 1;
+	}
+
+	if( append_mode && !(argc-woptind) )
+	{
+		sb_append2( sb_err,
+                    argv[0],
+                    BUILTIN_ERR_COMBO,
+                    L",\n insertion mode switches can not be used when not in insertion mode",
+                    0 );
+        builtin_print_help( argv[0], sb_err );		
+        return 1;
+	}
+
+	/*
+	  Set default modes
+	*/
+	if( !append_mode )
+	{
+		append_mode = REPLACE_MODE;
+	}
+	
+	if( !buffer_part )
+	{
+		buffer_part = STRING_MODE;
+	}
+		
+	wchar_t *begin, *end;
+	
+	switch( buffer_part )
+	{
+		case STRING_MODE:
+		{			
+			begin = reader_get_buffer();
+			end = begin+wcslen(begin);
+			break;			
+		}
+
+		case PROCESS_MODE:
+		{
+			reader_current_process_extent( &begin, &end );
+			break;
+		}
+		
+		case JOB_MODE:
+		{
+			reader_current_job_extent( &begin, &end );					
+			break;			
+		}
+		
+		case TOKEN_MODE:
+		{
+			reader_current_token_extent( &begin, &end, 0, 0 );
+			break;			
+		}
+				
+	}
+
+	switch(argc-woptind)
+	{
+		case 0:
+		{					
+			write_part( begin, end, cut_at_cursor, tokenize );
+			break;
+		}
+		
+		case 1:
+		{
+			replace_part( begin, end, argv[woptind], append_mode );
+			break;					
+		}		
+	}
+	
+	return 0;
+}

builtin_help.h

@@ -0,0 +1,25 @@
+/** \file builtin_help.h 
+
+    Prototypes for functions for printing usage information of builtin commands. The
+	corresponding .c file is automatically generated by combining the
+	builtin_help.hdr file with doxygen output.
+*/
+/**
+   Return the help text for the specified builtin command. Use
+   non-wide characters since wide characters have some issues with
+   string formating escape sequences sometimes.
+
+   \param cmd The command for which to obtain help text
+*/
+char *builtin_help_get( wchar_t *cmd );
+
+/**
+   Initialize builtin help data
+*/
+void builtin_help_init();
+
+/**
+   Destory builtin help data
+*/
+void builtin_help_destroy();
+

builtin_help.hdr

@@ -0,0 +1,36 @@
+/** \file builtin_help.c
+
+    Functions for printing usage information of builtin commands. This
+	file is automatically generated from the file builtin_help.hdr and
+	various help files in the doc_src directory.
+*/
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <strings.h>
+#include <wchar.h>
+
+#include "config.h"
+#include "util.h"
+#include "common.h"
+#include "builtin_help.h"
+
+/**
+	Hashtable storing the help text
+*/
+static hash_table_t tbl;
+
+char *builtin_help_get( wchar_t *cmd )
+{
+	return (char *)hash_get( &tbl, (void *)cmd );
+}
+
+void builtin_help_destroy()
+{
+	hash_destroy( &tbl );
+}
+
+
+void builtin_help_init()
+{
+	hash_init( &tbl, &hash_wcs_func, &hash_wcs_cmp );

builtin_set.c

@@ -0,0 +1,585 @@
+/** \file builtin_set.c Functions defining the set builtin
+
+Functions used for implementing the set builtin. 
+
+*/
+#include <stdlib.h>
+#include <stdio.h>
+#include <wchar.h>
+#include <wctype.h>
+#include <sys/types.h>
+#include <termios.h>
+#include <signal.h>
+
+#include "config.h"
+#include "util.h"
+#include "builtin.h"
+#include "env.h"
+#include "expand.h"
+#include "common.h"
+#include "wgetopt.h"
+#include "proc.h"
+#include "parser.h"
+
+/** 
+	Extract the name from a destination argument of the form name[index1 index2...]
+*/
+static int parse_fill_name( string_buffer_t *name, 
+							const wchar_t *src) 
+{
+
+	if (src == 0) 
+	{
+		return 0;
+	}
+	
+	while (iswalnum(*src) || *src == L'_')
+	{
+		sb_append_char(name, *src++);
+	}
+	
+	if (*src != L'[' && *src != L'\0') 
+	{
+	
+		sb_append(sb_err, L"set: Invalid character in variable name: ");
+		sb_append_char(sb_err, *src);
+		sb_append2(sb_err, L"\n", 				  parser_current_line(), L"\n", 0 );
+//		builtin_print_help( L"set", sb_err );
+
+		return -1;
+	}
+	else 
+	{
+		return 0;
+	}
+}
+
+
+/** 
+	Extract indexes from a destination argument of the form name[index1 index2...]
+*/
+static int parse_fill_indexes( array_list_t *indexes,
+							   const wchar_t *src)
+{
+	int count = 0;
+
+	if (src == 0)
+	{
+		return 0;
+	}
+	
+	while (*src != L'\0' && (iswalnum(*src) || *src == L'_'))
+	{
+		src++;
+	}
+	
+	if (*src == L'\0') 
+	{
+		return 0;
+	}
+	
+	if (*src++ != L'[')
+	{
+		return -1;
+	}
+	
+	while (iswblank(*src)) 
+	{
+		src++;
+	}
+	
+	while (*src != L']') 
+	{
+		wchar_t *end;
+		long l_ind = wcstol(src, &end, 10);
+		if (end == src) 
+		{
+			wchar_t sbuf[256];
+			swprintf(sbuf, 255, L"Invalid index starting at %ls\n", src);
+			sb_append(sb_err, sbuf);
+			return -1;
+		}
+
+		int *ind = (int *) calloc(1, sizeof(int));
+		*ind = (int) l_ind;
+		al_push(indexes, ind);
+		src = end;
+		count++;
+		while (iswblank(*src)) src++;
+	}
+
+	return count;
+}
+
+
+/**
+   Update a list by writing the specified values at the specified indexes
+*/
+static int update_values( array_list_t *list, 
+						  array_list_t *indexes,
+						  array_list_t *values ) 
+{
+	int i;
+
+	//fwprintf(stderr, L"Scan complete\n");
+	/* Replace values where needed */
+	for( i = 0; i < al_get_count(indexes); i++ ) 
+	{
+		int ind = *(int *) al_get(indexes, i) - 1;
+		void *new = (void *) al_get(values, i);
+		if (al_get(list, ind) != 0) 
+		{
+			free((void *) al_get(list, ind));
+		}
+		al_set(list, ind, new != 0 ? wcsdup(new) : wcsdup(L""));
+	}
+  
+	return al_get_count(list);
+}
+
+
+/**
+   Return 1 if an array list of int* pointers contains the specified
+   value, 0 otherwise
+*/
+static int al_contains_int( array_list_t *list,
+							int val)
+{
+	int i;
+
+	for (i = 0; i < al_get_count(list); i++) 
+	{
+		int *current = (int *) al_get(list, i);
+		if (current != 0 && *current == val) 
+		{
+			return 1;
+		}
+	}
+	
+	return 0;
+}
+
+
+/**
+   Erase from a list values at specified indexes 
+*/
+static int erase_values(array_list_t *list, array_list_t *indexes) 
+{
+	int i;
+	array_list_t result;
+
+	//fwprintf(stderr, L"Starting with %d\n", al_get_count(list));
+
+	al_init(&result);
+
+	for (i = 0; i < al_get_count(list); i++) 
+	{
+		if (!al_contains_int(indexes, i + 1)) 
+		{
+			al_push(&result, al_get(list, i));
+		}
+		else 
+		{
+			free((void *) al_get(list, i));
+		}
+	}
+	
+	al_truncate(list,0);	
+	al_push_all( list, &result );
+	al_destroy(&result);
+
+	//fwprintf(stderr, L"Remaining: %d\n", al_get_count(&result));
+
+	return al_get_count(list);    
+}
+
+
+/**
+   Fill a string buffer with values from a list, using ARRAY_SEP_STR to separate them 
+*/
+static int fill_buffer_from_list(string_buffer_t *sb, array_list_t *list) 
+{
+	int i;
+
+	for (i = 0; i < al_get_count(list); i++) 
+	{
+		wchar_t *v = (wchar_t *) al_get(list, i);
+		if (v != 0) 
+		{
+			// fwprintf(stderr, L".\n");
+			// fwprintf(stderr, L"Collecting %ls from %d\n", v, i);
+			sb_append(sb, v);
+		}    
+		if (i < al_get_count(list) - 1)
+			sb_append(sb, ARRAY_SEP_STR);
+	}
+	return al_get_count(list);
+}
+
+
+/**
+   Print the names of all environment variables in the scope, with or without values,
+   with or without escaping
+*/
+static void print_variables(int include_values, int escape, int scope) 
+{
+	array_list_t names;
+	wchar_t **names_arr;
+	int i;
+  
+	al_init( &names );
+	env_get_names( &names, scope );
+	names_arr = list_to_char_arr( &names );
+	qsort( names_arr, 
+		   al_get_count( &names ),
+		   sizeof(wchar_t *),
+		   (int (*)(const void *, const void *))&wcsfilecmp );
+  
+	for( i = 0; i < al_get_count(&names); i++ )
+	{
+		wchar_t *key = names_arr[i];
+		/* Why does expand_escape free its argument ?! */
+		wchar_t *e_key = escape ? expand_escape(wcsdup(key), 1) : wcsdup(key);
+		sb_append(sb_out, e_key);
+
+		if( include_values ) 
+		{
+			wchar_t *value = env_get(key);
+			wchar_t *e_value = escape ? expand_escape_variable(value) : wcsdup(value);
+			sb_append2(sb_out, L" ", e_value, 0);
+			free(e_value);
+		}
+
+		sb_append(sb_out, L"\n");
+		free(e_key);
+	}
+  
+	free(names_arr);
+	al_destroy(&names);
+}
+
+/**
+   The set builtin. Creates, updates and erases environment variables and environemnt variable arrays. 
+*/
+
+int builtin_set( wchar_t **argv ) 
+{
+	const static struct woption
+		long_options[] = 
+		{
+			{ 
+				L"export", no_argument, 0, 'x' 
+			},
+			{ 
+				L"global", no_argument, 0, 'g' 
+			},
+			{ 
+				L"local", no_argument, 0, 'l'  
+			},
+			{ 
+				L"erase", no_argument, 0, 'e'  
+			},
+			{ 
+				L"names", no_argument, 0, 'n' 
+			} ,
+			{ 
+				L"unexport", no_argument, 0, 'u' 
+			} ,
+			{ 
+				L"universal", no_argument, 0, 'U' 
+			} ,
+			{ 
+				0, 0, 0, 0 
+			}
+		}
+	;
+	
+	wchar_t short_options[] = L"xglenuU";
+
+	int argc = builtin_count_args(argv);
+
+	/*
+	  Flags to set the work mode
+	*/
+	int local = 0, global = 0, export = 0;
+	int erase = 0, list = 0, unexport=0;
+	int universal = 0;
+	
+
+	/*
+	  Variables used for performing the actual work
+	*/
+	wchar_t *dest = 0;
+	array_list_t values;
+	string_buffer_t name_sb;
+	int retcode=0;
+	wchar_t *name;
+	array_list_t indexes;
+	int retval;
+
+
+	/* Parse options to obtain the requested operation and the modifiers */
+	woptind = 0;
+	while (1) 
+	{
+		int c = wgetopt_long(argc, argv, short_options, long_options, 0);
+
+		if (c == -1) 
+		{
+			break;
+		}
+    
+		switch(c) 
+		{
+			case 0:
+				break;
+			case 'e':
+				erase = 1;
+				break;
+			case 'n':
+				list = 1;
+				break;
+			case 'x':
+				export = 1;
+				break;
+			case 'l':
+				local = 1;
+				break;
+			case 'g':
+				global = 1;
+				break;
+			case 'u':
+				unexport = 1;
+				break;
+			case 'U':
+				universal = 1;
+				break;
+			case '?':
+				return 1;
+			default:
+				break;
+		}
+	}
+
+	/* Check operation and modifiers sanity */
+	if( erase && list ) 
+	{
+		sb_append2(sb_err,
+				  argv[0],
+				  BUILTIN_ERR_COMBO,
+				  L"\n",
+				  parser_current_line(),
+				  L"\n",
+				  0);
+		builtin_print_help( argv[0], sb_err );
+		return 1;
+	}
+
+	if( local + global + universal > 1 ) 
+	{
+		sb_printf( sb_err,
+				   L"%ls%ls\n%ls\n",
+				   argv[0],
+				   BUILTIN_ERR_GLOCAL,
+				   parser_current_line() );
+		builtin_print_help( argv[0], sb_err );
+		return 1;
+	}
+
+	if( export && unexport ) 
+	{
+		sb_append2(sb_err,
+				   argv[0],
+				   BUILTIN_ERR_EXPUNEXP,
+				   L"\n",
+				   parser_current_line(),
+				   L"\n", 
+				   0);
+		builtin_print_help( argv[0], sb_err );
+		return 1;
+	}
+
+	/* Parse destination */
+	if( woptind < argc ) 
+	{
+		dest = wcsdup(argv[woptind++]);
+		//fwprintf(stderr, L"Dest: %ls\n", dest);
+	}
+
+	/* Parse values */
+	// wchar_t **values = woptind < argc ? (wchar_t **) calloc(argc - woptind, sizeof(wchar_t *)) : 0;
+
+	al_init(&values);
+	while( woptind < argc ) 
+	{
+		al_push(&values, argv[woptind++]);
+		//    fwprintf(stderr, L"Val: %ls\n", argv[woptind - 1]);
+	}
+
+	/* Extract variable name and indexes */
+
+	sb_init(&name_sb);
+	retval = parse_fill_name(&name_sb, dest);
+
+	
+	if( retval < 0 ) 
+		retcode=1;
+
+	if( !retcode )
+	{
+		name = (wchar_t *) name_sb.buff;
+		//fwprintf(stderr, L"Name is %ls\n", name);
+
+		al_init(&indexes);
+		retval = parse_fill_indexes(&indexes, dest);
+		if (retval < 0) 
+			retcode = 1;
+	}
+	
+	if( !retcode )
+	{
+		
+		int i;
+		int finished=0;
+		
+		/* Do the actual work */
+		int scope = (local ? ENV_LOCAL : 0) | (global ? ENV_GLOBAL : 0) | (export ? ENV_EXPORT : 0) | (unexport ? ENV_UNEXPORT : 0) | (universal ? ENV_UNIVERSAL:0) | ENV_USER; 
+		if( list ) 
+		{
+			/* Maybe we should issue an error if there are any other arguments */
+			print_variables(0, 0, scope);
+			finished=1;			
+		} 
+		
+		if( (!finished ) && 
+			 (name == 0 || wcslen(name) == 0))
+		{
+			/* No arguments -- display name & value for all variables in scope */
+			if( erase ) 
+			{
+				sb_append2( sb_err,
+							argv[0],
+							L": Erase needs a variable name\n", 
+							parser_current_line(), 
+							L"\n",
+							0 );
+				builtin_print_help( argv[0], sb_err );
+				retcode = 1;
+			}
+			else 
+			{
+				print_variables( 1, 1, scope );
+			}
+
+			finished=1;			
+		} 
+
+
+		if( !finished )
+		{
+			if( al_get_count( &values ) == 0 && 
+				al_get_count( &indexes ) == 0 &&
+				!erase &&
+				!list )
+			{
+				/*
+				  Only update the variable scope
+				*/
+				env_set( name, 0, scope );
+				finished = 1;
+			}
+		}
+		
+		if( !finished )
+		{
+			/* There are some arguments, we have at least a variable name */
+			if( erase && al_get_count(&values) != 0 ) 
+			{
+				sb_append2( sb_err, 
+							argv[0],
+							L": Values cannot be specfied with erase\n",
+							parser_current_line(),
+							L"\n",
+							0 );
+				builtin_print_help( argv[0], sb_err );
+				retcode = 1;
+			} 
+			else
+			{    
+				/* All ok, we can alter the specified variable */
+				array_list_t val_l;
+				al_init(&val_l);
+
+				void *old=0;
+				
+				if (al_get_count(&indexes) == 0) 
+				{
+					/* We will act upon the entire variable */
+	
+ 					al_push( &val_l, wcsdup(L"") );
+					old = val_l.arr;
+					
+					/* Build indexes for all variable or all new values */
+					int end_index = erase ? al_get_count(&val_l) : al_get_count(&values);
+					for (i = 0; i < end_index; i++) 
+					{
+						int *ind = (int *) calloc(1, sizeof(int));
+						*ind = i + 1;
+						al_push(&indexes, ind);
+					}
+				}
+				else 
+				{
+					/* We will act upon some specific indexes */
+					expand_variable_array( env_get(name), &val_l );
+				}
+				
+				string_buffer_t result_sb;
+				sb_init(&result_sb);
+				if (erase) 
+				{
+					int rem = erase_values(&val_l, &indexes);
+					if (rem == 0) 
+					{
+						env_remove(name, ENV_USER);
+					}
+					else 
+					{
+						fill_buffer_from_list(&result_sb, &val_l);
+						env_set(name, (wchar_t *) result_sb.buff, scope);
+					}     
+				}
+				else
+				{
+					
+					update_values( &val_l, 
+								   &indexes,
+								   &values );
+
+					fill_buffer_from_list( &result_sb,
+										   &val_l );
+
+					env_set(name,
+							(wchar_t *) result_sb.buff,
+							scope);
+				}
+				
+				al_foreach( &val_l, (void (*)(const void *))&free );
+				al_destroy(&val_l);
+				sb_destroy(&result_sb); 
+			}
+		}
+
+		al_foreach( &indexes, (void (*)(const void *))&free );
+		al_destroy(&indexes);
+	}
+	
+/* Common cleanup */
+//fwprintf(stderr, L"Cleanup\n");
+	free(dest);		
+	sb_destroy(&name_sb);
+	al_destroy( &values );
+	
+	return retcode;
+}
+

common.h

@@ -0,0 +1,248 @@
+/** \file common.h
+	Prototypes for various functions, mostly string utilities, that are used by most parts of fish.
+*/
+
+
+/**
+   Under curses, tputs expects an int (*func)(char) as its last parameter, but in ncurses, tputs expects a int (*func)(int) as its last parameter. tputs_arg_t is defined to always be what tputs expects. Hopefully.
+*/
+
+#ifdef NCURSES_VERSION
+typedef int tputs_arg_t;
+#else
+typedef char tputs_arg_t;
+#endif
+
+/**
+   Maximum number of bytes in a utf-8 character
+*/
+#define MAX_UTF8_BYTES 6
+
+/**
+   Amount of debug info to show. Higher level means more debug info will be displayed
+*/
+#define DEBUG_LEVEL 1
+
+/**
+   Color code for set_color. Does not update the color.
+*/
+
+#define FISH_COLOR_IGNORE -1
+
+/**
+   Color code for set_color. Sets the default color.
+*/
+#define FISH_COLOR_RESET -2
+
+/** Save the shell mode on startup so we can restore them on exit */
+extern struct termios shell_modes;      
+
+/**
+   The character to use where the text has been truncated. Is an ellipsis on unicode system and a $ on other systems.
+*/
+extern wchar_t ellipsis_char;
+
+/**
+   The maximum number of charset convertion errors to report
+*/
+extern int error_max;
+
+/**
+   Profiling flag. True if commands should be profiled.
+*/
+extern char *profile;
+
+/**
+   Name of the current program. Should be set at startup. Used by the
+   debug function.
+*/
+extern wchar_t *program_name;
+/**
+   Take an array_list_t containing wide strings and converts them to a wchar_t **.
+*/
+wchar_t **list_to_char_arr( array_list_t *l );
+
+/**
+   Read a line from the stream f into the buffer buff of length len. If
+   buff is to small, it will be reallocated, and both buff and len will
+   be updated to reflect this. Returns the number of bytes read or -1
+   on failiure. 
+
+   If the carriage return character is encountered, it is
+   ignored. fgetws() considers the line to end if reading the file
+   results in either a newline (L'\n') character, the null (L'\\0')
+   character or the end of file (WEOF) character.
+*/
+int fgetws2( wchar_t **buff, int *len, FILE *f );
+
+/**
+   Sorts a list of wide strings according to the wcsfilecmp-function from the util library
+*/
+void sort_list( array_list_t *comp );
+
+/**
+   Returns a newly allocated wide character string equivalent of the specified multibyte character string
+*/
+wchar_t *str2wcs( const char *in );
+
+/**
+   Returns a newly allocated multibyte character string equivalent of the specified wide character string
+*/
+char *wcs2str( const wchar_t *in );
+
+/**
+   Returns a newly allocated wide character string array equivalent of the specified multibyte character string array
+*/
+char **wcsv2strv( const wchar_t **in );
+
+/**
+   Returns a newly allocated multibyte character string array equivalent of the specified wide character string array
+*/
+wchar_t **strv2wcsv( const char **in );
+
+/**
+   Returns a newly allocated concatenation of the specified wide character strings
+*/
+wchar_t *wcsdupcat( const wchar_t *a, const wchar_t *b );
+
+/**
+   Returns a newly allocated concatenation of the specified wide character strings. The last argument must be a null pointer.
+*/
+wchar_t *wcsdupcat2( const wchar_t *a, ... );
+
+/**
+   Returns a newly allocated wide character string wich is a copy of the string in, but of length c or shorter. The returned string is always null terminated, and the null is not included in the string length.
+*/
+wchar_t *wcsndup( const wchar_t *in, int c );
+
+/**
+   Converts from wide char to digit in the specified base. If d is not
+   a valid digit in the specified base, return -1.
+*/
+long convert_digit( wchar_t d, int base );
+
+
+/**
+   Convert a wide character string to a number in the specified
+   base. This functions is the wide character string equivalent of
+   strtol. For bases of 10 or lower, 0..9 are used to represent
+   numbers. For bases below 36, a-z and A-Z are used to represent
+   numbers higher than 9. Higher bases than 36 are not supported.
+*/
+long wcstol(const wchar_t *nptr,
+			wchar_t **endptr,
+			int base);
+
+size_t
+wcslcat(wchar_t *dst, const wchar_t *src, size_t siz);
+
+size_t
+wcslcpy(wchar_t *dst, const wchar_t *src, size_t siz);
+
+/**
+   Create a dublicate string. Wide string version of strdup.
+*/
+wchar_t *wcsdup(const wchar_t *in);
+
+/**
+   Case insensitive string compare function. Wide string version of
+   strcasecmp.
+
+   This implementation of wcscasecmp does not take into account
+   esoteric locales where uppercase and lowercase do not cleanly
+   transform between each other. Hopefully this should be fine since
+   fish only uses this function with one of the strings supplied by
+   fish and guaranteed to be a sane, english word.
+*/
+int wcscasecmp( const wchar_t *a, const wchar_t *b );
+
+/**
+   Test if the given string is a valid variable name
+*/
+
+int wcsvarname( wchar_t *str );
+
+/**
+   The prototype for this function is missing in some libc
+   implementations. Fish has a fallback implementation in case the
+   implementation is missing altogether.
+*/
+int wcwidth( wchar_t c );
+
+
+/**
+   A wcswidth workalike. Fish uses this since the regular wcswidth seems flaky.
+*/
+int my_wcswidth( const wchar_t *c );
+
+/**
+   This functions returns the end of a quoted substring. It can handle nested single and double quotes.
+*/
+wchar_t *quote_end( const wchar_t *in );
+
+/**
+   A call to this function will reset the error counter. Some
+   functions print out non-critical error messages. These should check
+   the error_count before, and skip printing the message if
+   MAX_ERROR_COUNT messages have been printed. The error_reset()
+   should be called after each interactive command executes, to allow
+   new messages to be printed.
+*/
+void error_reset();
+
+/**
+   Set the locale, also change the ellipsis character
+*/
+void fish_setlocale(int category, const wchar_t *locale);
+
+/**
+   Checks if \c needle is included in the list of strings specified
+
+   \param needle the string to search for in the list 
+*/
+int contains_str( const wchar_t *needle, ... );
+
+/**
+   Call read while blocking the SIGCHLD signal. Should only be called
+   if you _know_ there is data available for reading.
+*/
+int read_blocked(int fd, void *buf, size_t count);
+
+/**
+   This is for writing process notification messages. Has to write to
+   stdout, so clr_eol and such functions will work correctly. Not an
+   issue since this function is only used in interactive mode anyway.
+*/
+int writeb( tputs_arg_t b );
+
+void die_mem();
+
+/**
+  Clean up 
+*/
+void common_destroy();
+
+/**
+   Issue a debug message
+   
+   \param level the priority of the message. Lower number means higher priority. Messages with too high priority number will be discarded.
+   \param the message. 
+*/
+void debug( int level, wchar_t *msg, ... );
+
+/**
+   Replace special characters with escape sequences. Newline is
+   replaced with \n, etc. 
+
+   \param in The string to be escaped
+   \param escape_all Whether all characters wich hold special meaning in fish (Pipe, semicolon, etc,) should be escaped, or only unprintable characters
+   \return The escaped string, or 0 if there is not enough memory
+*/
+
+wchar_t *escape( wchar_t *in, 
+				 int escape_all );
+
+wchar_t *unescape( wchar_t * in, int escape_special );
+
+void block();
+void unblock();

complete.h

@@ -0,0 +1,155 @@
+/** \file complete.h
+	Prototypes for functions related to tab-completion.
+
+	These functions are used for storing and retrieving tab-completion data, as well as for performing tab-completion.
+*/
+
+/** Use all completions */
+#define SHARED 0
+/** Do not use file completion */
+#define NO_FILES 1
+/** Require a parameter after completion */
+#define NO_COMMON 2
+/** Only use the argument list specifies with completion after option. This is the same as (NO_FILES & NO_COMMON) */
+#define EXCLUSIVE 3
+
+/** Command is a path */
+#define PATH 1
+/** Command is not a path */
+#define COMMAND 0
+
+/** Separateor between completion and description*/
+#define COMPLETE_SEP L'\004'
+/** Separateor between completion and description*/
+#define COMPLETE_SEP_STR L"\004"
+
+/**
+   Character that separates the completion and description on programmable completions
+*/
+#define PROG_COMPLETE_SEP L'\t'
+
+/**
+  Initializes various structures used for tab-completion.
+*/
+void complete_init();
+
+/**
+   Destroys various structures used for tab-completion and free()s the memory used by them.
+*/
+void complete_destroy();
+
+/**
+
+  Add a completion. 
+
+  Values are copied and should be freed by the caller.
+
+  Examples: 
+  
+  The command 'gcc -o' requires that a file follows it, so the
+  NO_COMMON option is suitable. This can be done using the following
+  line:
+  
+  complete -c gcc -s o -r
+
+  The command 'grep -d' required that one of the strings 'read',
+  'skip' or 'recurse' is used. As such, it is suitable to specify that
+  a completion requires one of them. This can be done using the
+  following line:
+
+  complete -c grep -s d -x -a "read skip recurse"
+
+
+  \param cmd Command to complete.
+  \param cmd_type If cmd_type is PATH, cmd will be interpreted as the absolute
+  path of the program (optionally containing wildcards), otherwise it
+  will be interpreted as the command name.
+  \param short_opt The single character name of an option. (-a is a short option, --all and  -funroll are long options)
+  \param long_opt The multi character name of an option. (-a is a short option, --all and  -funroll are long options)
+  \param long_mode Whether to use old style, single dash long options. 
+  \param result_mode Whether to search further completions when this
+  completion has been succesfully matched. If result_mode is SHARED,
+  any other completions may also be used. If result_mode is NO_FILES,
+  file completion should not be used, but other completions may be
+  used. If result_mode is NO_COMMON, on option may follow it - only a
+  parameter. If result_mode is EXCLUSIVE, no option may follow it, and
+  file completion is not performed.
+  \param comp A space separated list of completions which may contain subshells.
+  \param desc A description of the completion.
+  \param authorative Whether there list of completions for this command is complete. If true, any options not matching one of the provided options will be flagged as an error by syntax highlighting.
+  \param condition a command to be run to check it this completion should be used. If \c condition is empty, the completion is always used.
+
+*/
+void complete_add( const wchar_t *cmd, 
+				   int cmd_type, 
+				   wchar_t short_opt,
+				   const wchar_t *long_opt,
+				   int long_mode, 
+				   int result_mode, 
+				   int authorative,
+				   const wchar_t *condition,
+				   const wchar_t *comp,
+				   const wchar_t *desc ); 
+
+/**
+  Remove a previously defined completion
+*/
+void complete_remove( const wchar_t *cmd, 
+					  int cmd_type, 
+					  wchar_t short_opt,
+					  const wchar_t *long_opt );
+
+/**
+  Find all completions of the command cmd, insert them into out. The
+  caller must free the variables returned in out.  The results are
+  returned in the array_list_t 'out', in the format of wide character
+  strings, with each element consisting of a suggested completion and
+  a description of what kind of object this completion represents,
+  separated by a separator of type COMPLETE_SEP.
+
+  Values returned by this function should be freed by the caller.
+*/
+void complete( const wchar_t *cmd, array_list_t *out );
+
+/**
+   Print a list of all current completions into the string_buffer_t. 
+
+   \param out The string_buffer_t to write completions to
+*/
+void complete_print( string_buffer_t *out );
+
+/**
+   Obtain a description string for the file specified by the filename.
+
+   The returned value is a string constant and should not be freed.
+
+   \param filename The file for which to find a description string
+*/
+const wchar_t *complete_get_desc( const wchar_t *filename );
+
+/**
+   Tests if the specified option is defined for the specified command
+*/
+int complete_is_valid_option( const wchar_t *str, 
+							  const wchar_t *opt, 
+							  array_list_t *errors );
+
+/**
+   Tests if the specified argument is valid for the specified option
+   and command
+*/
+int complete_is_valid_argument( const wchar_t *str, 
+								const wchar_t *opt, 
+								const wchar_t *arg );
+
+
+/**
+   Load command-specific completions for the specified command. This
+   is done automatically whenever completing any given command, so
+   there is no need to call this except in the case of completions
+   with internal dependencies.
+
+   \param cmd the command for which to load command-specific completions
+   \param reload should the commands completions be reloaded, even if they where previously loaded. (This is set to true on actual completions, so that changed completion are updated in running shells)
+*/
+void complete_load( wchar_t *cmd, int reload );

config.h.in

@@ -0,0 +1,82 @@
+/* config.h.in.  Generated from configure.ac by autoheader.  */
+
+/* CPU type */
+#undef CPU
+
+/* Documentation directory */
+#undef DOCDIR
+
+/* Define to 1 if you have the `futimes' function. */
+#undef HAVE_FUTIMES
+
+/* Define to 1 if you have the <getopt.h> header file. */
+#undef HAVE_GETOPT_H
+
+/* Define to 1 if you have the <inttypes.h> header file. */
+#undef HAVE_INTTYPES_H
+
+/* Define to 1 if you have the <memory.h> header file. */
+#undef HAVE_MEMORY_H
+
+/* Define to 1 if you have the <ncurses.h> header file. */
+#undef HAVE_NCURSES_H
+
+/* Define to 1 if you have the <stdint.h> header file. */
+#undef HAVE_STDINT_H
+
+/* Define to 1 if you have the <stdlib.h> header file. */
+#undef HAVE_STDLIB_H
+
+/* Define to 1 if you have the <strings.h> header file. */
+#undef HAVE_STRINGS_H
+
+/* Define to 1 if you have the <string.h> header file. */
+#undef HAVE_STRING_H
+
+/* Define to 1 if you have the <sys/stat.h> header file. */
+#undef HAVE_SYS_STAT_H
+
+/* Define to 1 if you have the <sys/types.h> header file. */
+#undef HAVE_SYS_TYPES_H
+
+/* Define to 1 if you have the <termio.h> header file. */
+#undef HAVE_TERMIO_H
+
+/* Define to 1 if you have the <unistd.h> header file. */
+#undef HAVE_UNISTD_H
+
+/* Define to 1 if you have the `wcswidth' function. */
+#undef HAVE_WCSWIDTH
+
+/* Define to 1 if you have the `wcwidth' function. */
+#undef HAVE_WCWIDTH
+
+/* Define to 1 if you have the `wprintf' function. */
+#undef HAVE_WPRINTF
+
+/* Define to 1 if you have the file `AC_File'. */
+#undef HAVE__PROC_SELF_STAT
+
+/* Define to the address where bug reports for this package should be sent. */
+#undef PACKAGE_BUGREPORT
+
+/* Define to the full name of this package. */
+#undef PACKAGE_NAME
+
+/* Define to the full name and version of this package. */
+#undef PACKAGE_STRING
+
+/* Define to the one symbol short name of this package. */
+#undef PACKAGE_TARNAME
+
+/* Define to the version of this package. */
+#undef PACKAGE_VERSION
+
+/* Installation directory */
+#undef PREFIX
+
+/* Define to 1 if you have the ANSI C header files. */
+#undef STDC_HEADERS
+
+/* Evil kludge to get Power based machines to work */
+#undef TPUTS_KLUDGE

configure.ac

@@ -0,0 +1,90 @@
+# Process this file with autoconf to produce a configure script.
+AC_INIT(fish,1.14.0,axel@liljencrantz.se)
+
+AC_CANONICAL_TARGET
+
+if test $target_cpu = powerpc; then
+   AC_DEFINE([TPUTS_KLUDGE],[1],[Evil kludge to get Power based machines to work])
+fi
+
+AC_DEFINE_UNQUOTED([CPU],[L"$target_cpu"],[CPU type])
+
+AC_CONFIG_HEADERS(config.h)
+
+# Checks for programs.
+AC_PROG_CC
+AC_PROG_CPP
+AC_PROG_INSTALL
+#AC_ISC_POSIX
+#AC_PROG_MAKE_SET
+
+# Optionally drop xsel
+AC_ARG_WITH( xsel, 
+			 AC_HELP_STRING([--without-xsel],
+             [do not build the xsel program needed for X clipboard integration]),
+			 [xsel=$withval],
+			 [xsel=with_xsel] )
+
+if [[ "$xsel" = "with_xsel" ]]; then
+    AC_SUBST( XSEL,[xsel-0.9.6/xsel])
+    AC_SUBST( XSEL_MAN,[xsel.1x])
+    AC_SUBST( XSEL_MAN_PATH,[xsel-0.9.6/xsel.1x])
+else
+    AC_SUBST( XSEL,[ ])
+    AC_SUBST( XSEL_MAN,[ ])
+    AC_SUBST( XSEL_MAN_PATH,[ ])
+fi
+
+if [[ "$prefix" = NONE ]]; then
+   	AC_DEFINE_UNQUOTED( [PREFIX], L"/usr/local", [Installation directory])
+	AC_SUBST( PREFIX, /usr/local)
+   	AC_SUBST(sysconfdir,[/etc])
+	export prefix=/usr/local
+else
+	AC_DEFINE_UNQUOTED( [PREFIX], L"$prefix", [Installation directory])
+	AC_SUBST( PREFIX, [$prefix])
+   	AC_SUBST(sysconfdir,[/etc])
+fi
+
+if echo $prefix | grep \^$HOME >/dev/null; then
+  	AC_SUBST(sysconfdir,[$HOME])
+	AC_SUBST(fishdir,[/.fish.d])
+	AC_SUBST(fishfile,[/.fish])
+	AC_SUBST(fishinputfile,[/.fish_inputrc])
+	AC_MSG_NOTICE(["Install in $HOME"])
+else
+	AC_SUBST(fishdir,[/fish.d])
+	AC_SUBST(fishfile,[/fish])
+	AC_SUBST(fishinputfile,[/fish_inputrc])
+fi
+
+AC_ARG_VAR( [docdir], [Documentation direcotry] )
+
+if test -z $docdir; then
+   AC_SUBST(docdir,[$datadir/doc/fish])
+fi
+
+AC_DEFINE_UNQUOTED( DOCDIR, [L"$(eval echo $docdir)"], [Documentation directory] )
+
+# See if Linux procfs is present
+AC_CHECK_FILES([/proc/self/stat])
+
+# See if NetBSD pkgsrc is installed
+AC_CHECK_FILE([/usr/pkg/lib],[AC_SUBST(LIBDIR,[-L/usr/pkg/lib\ -R/usr/pkg/lib])])
+AC_CHECK_FILE([/usr/pkg/include],[AC_SUBST(INCLUDEDIR,[-I/usr/pkg/include])])
+
+AC_CHECK_FUNCS( [wprintf futimes wcwidth wcswidth] ) 
+AC_CHECK_HEADERS([getopt.h termio.h])
+
+
+
+# Check if we have ncurses, and use it rather than curses if possible.
+AC_CHECK_HEADERS([ncurses.h],[AC_SUBST(CURSESLIB,[ncurses]) AC_DEFINE(HAVE_NCURSES_H)],[AC_SUBST(CURSESLIB,[curses])])
+
+#Force use of ncurses if it is avialable via NetBSD pkgsrc. This is an
+#ugly kludge to force NetBSD to use ncurses, since NetBSDs own version
+#does not properly support terminfo.
+AC_CHECK_FILE([/usr/pkg/include/ncurses.h],[AC_SUBST(CURSESLIB,[ncurses]) AC_DEFINE(HAVE_NCURSES_H)])
+
+AC_CONFIG_FILES([Makefile fish.spec doc_src/fish.1 doc_src/Doxyfile init/fish])
+AC_OUTPUT

count.c

@@ -0,0 +1,22 @@
+/** \file count.c
+   The length command, used for determining the number of items in an
+   environment variable array.
+*/
+#include <stdlib.h>
+#include <stdio.h>
+
+/**
+   The main function. Does nothing but return the number of arguments.
+
+   This command, unlike all other fish commands, does not feature a -h
+   or --help option. This is because we want to avoid errors on arrays
+   that have -h or --help as entries, which is very common when
+   parsing options, etc. For this reason, the main fish binary does a
+   check and prints help usage if -h or --help is explicitly given to
+   the command, but not if it is the contents of a variable.
+*/
+int main( int argc, char **argv )
+{
+	printf( "%d\n", argc-1 );
+	return argc==1;
+}

env.c

@@ -0,0 +1,763 @@
+/** \file env.c
+	Functions for setting and getting environment variables.
+*/
+
+#include <stdlib.h>
+#include <wchar.h>
+#include <string.h>
+#include <stdio.h>
+#include <locale.h>
+#include <unistd.h>
+#include <signal.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <pwd.h>
+
+
+#if HAVE_NCURSES_H
+#include <ncurses.h>
+#else
+#include <curses.h>
+#endif
+
+#if HAVE_TERMIO_H
+#include <termio.h>
+#endif
+
+#include <term.h>
+
+#include "config.h"
+#include "util.h"
+#include "wutil.h"
+#include "proc.h"
+#include "common.h"
+#include "env.h"
+#include "sanity.h"
+#include "expand.h"
+#include "history.h"
+#include "reader.h"
+#include "parser.h"
+#include "env_universal.h"
+#include "env_universal.h"
+
+
+/**
+   Command used to start fishd
+*/
+#define FISHD_CMD L"if which fishd >/dev/null ^/dev/null; fishd ^/tmp/fish.%s.log; end"
+
+/**
+   At init, we read all the environment variables from this array 
+*/
+extern char **environ;
+
+static int c1=0;
+
+/**
+   Struct representing one level in the function variable stack
+*/
+typedef struct env_node
+{
+	/** 
+		Variable table 
+	*/
+	hash_table_t env;
+	/** 
+		Does this node imply a new variable scope? If yes, all
+		non-global variables below this one in the stack are
+		invisible. If new_scope is set for the global variable node,
+		the universe will explode.
+	*/
+	int new_scope;
+	/**
+	   Does this node contain any variables which are exported to subshells
+	*/
+	int export;
+	
+	/** 
+		Pointer to next level
+	*/
+	struct env_node *next;
+}
+env_node_t;
+
+/**
+   A variable entry. Stores the value of a variable and whether it
+   should be exported. Obviously, it needs to be allocated large
+   enough to fit the value string.
+*/
+typedef struct var_entry
+{
+	int export; /**< Whether the variable should be exported */
+	wchar_t val[0]; /**< The value of the variable */
+}
+var_entry_t;
+
+/**
+   Top node on the function stack
+*/
+static env_node_t *top=0;
+
+/**
+   Bottom node on the function stack
+*/
+static env_node_t *global_env = 0;
+
+
+/**
+   Table for global variables
+*/
+static hash_table_t *global;
+
+/**
+   Table of variables that may not be set using the set command.
+*/
+static hash_table_t env_read_only;
+
+/**
+   Exported variable array used by execv
+*/
+static char **export_arr=0;
+
+/**
+   Flag for checking if we need to regenerate the exported variable
+   array
+*/
+static int has_changed = 1;
+
+/**
+   Number of variables marked for export. The actual number of
+   variables actually exported may be lower because of variable
+   scoping rules.
+*/
+static int export_count=0;
+
+
+/**
+   Free hash key and hash value
+*/
+static void clear_hash_entry( const void *key, const void *data )
+{
+	var_entry_t *entry = (var_entry_t *)data;	
+	if( entry->export )
+		has_changed = 1;
+	
+	free( (void *)key );
+	free( (void *)data );
+}
+
+/**
+   This stringbuffer is used to store the value of dynamically
+   generated variables, such as history.
+*/
+static string_buffer_t dyn_var;
+
+/**
+   Variable used by env_get_names to communicate auxiliary information
+   to add_key_to_hash
+*/
+static int get_names_show_exported;
+/**
+   Variable used by env_get_names to communicate auxiliary information
+   to add_key_to_hash
+*/
+static int get_names_show_unexported;
+
+/**
+   When fishd isn't started, this function is provided to
+   env_universal as a callback, it tries to start upå fishd. It's
+   implementation is a bit of a hack, since it just calls a bit of
+   shellscript, and the shell is not properly initialized ad this
+   point. Should be changed to deferr the evaluation until fish has
+   been properly initialized.
+*/
+static void start_fishd()
+{
+	string_buffer_t cmd;
+	struct passwd *pw;
+		
+	sb_init( &cmd );
+	pw = getpwuid(getuid());
+	
+	debug( 3, L"Spawning new copy of fishd" );
+	
+	if( !pw )
+	{
+		debug( 0, L"Could not get user information" );
+		return;
+	}
+	
+	sb_printf( &cmd, FISHD_CMD, pw->pw_name );
+	
+	eval( (wchar_t *)cmd.buff,
+		  0,
+		  TOP );
+	sb_destroy( &cmd );
+}
+
+void env_init()
+{
+	char **p;
+
+	sb_init( &dyn_var );
+	
+	/*
+	  These variables can not be altered directly by the user
+	*/
+	hash_init( &env_read_only, &hash_wcs_func, &hash_wcs_cmp );
+
+	hash_put( &env_read_only, L"status", L"" );
+	hash_put( &env_read_only, L"history", L"" );
+	hash_put( &env_read_only, L"_", L"" );
+	hash_put( &env_read_only, L"LINES", L"" );
+	hash_put( &env_read_only, L"COLUMNS", L"" );
+	hash_put( &env_read_only, L"PWD", L"" );
+
+	/*
+	  HOME should be writeable by root, since this is often a
+	  convenient way to install software.
+	*/
+	if( getuid() != 0 )
+		hash_put( &env_read_only, L"HOME", L"" );
+	
+	top = malloc( sizeof(env_node_t) );
+	top->next = 0;
+	top->new_scope = 0;
+	top->export=0;
+	hash_init( &top->env, &hash_wcs_func, &hash_wcs_cmp );
+	global_env = top;
+	global = &top->env;	
+
+	/*
+	  Import environment variables
+	*/
+	for( p=environ; *p; p++ )
+	{
+		wchar_t *key, *val;
+		wchar_t *pos;
+		
+		key = str2wcs(*p);
+		
+		if( !key )
+			continue;
+		
+		val = wcschr( key, L'=' );
+		
+		if( val == 0 )
+			env_set( key, L"", ENV_EXPORT );
+		else
+		{ 
+			*val = L'\0';
+			val++;
+			pos=val;
+			while( *pos )
+			{
+				if( *pos == L':' )
+					*pos = ARRAY_SEP;
+				pos++;
+			}
+//			fwprintf( stderr, L"Set $%ls to %ls\n", key, val );
+			
+			env_set( key, val, ENV_EXPORT | ENV_GLOBAL );
+		}		
+		free(key);
+	}		
+
+	env_universal_init( env_get( L"FISHD_SOKET_DIR"), env_get(L"USER"), &start_fishd );
+	
+}
+
+void env_destroy()
+{
+	char **ptr;
+
+	env_universal_destroy();
+//	fwprintf( stderr, L"Filled %d exported vars\n", c1 );
+	
+	sb_destroy( &dyn_var );
+	
+	while( &top->env != global )
+		env_pop();
+
+	hash_destroy( &env_read_only );
+	
+	hash_foreach( global, &clear_hash_entry );
+	hash_destroy( global );
+	free( top );
+	
+	if( export_arr != 0 )
+	{
+		for( ptr = export_arr; *ptr; ptr++ )
+			free( *ptr );
+
+		free( export_arr );
+	}
+}
+
+/**
+   Find the scope hashtable containing the variable with the specified
+   key
+*/
+static env_node_t *env_get_node( const wchar_t *key )
+{
+	var_entry_t* res;
+	env_node_t *env = top;
+
+
+	while( env != 0 )
+	{
+		res = (var_entry_t *) hash_get( &env->env, 
+										key );
+		if( res != 0 )
+		{
+			return env;
+		}
+
+		if( env->new_scope )
+			env = global_env;
+		else		
+			env = env->next;
+	}
+	
+	return 0;
+}
+	
+void env_set( const wchar_t *key, 
+			  const wchar_t *val, 
+			  int var_mode )
+{
+	int free_val = 0;
+	var_entry_t *entry;
+	env_node_t *node;
+	int has_changed_old = has_changed;
+	int has_changed_new = 0;
+	var_entry_t *e=0;
+	
+	
+	if( (var_mode & ENV_USER ) && 
+		hash_get( &env_read_only, key ) )
+	{
+		return;
+	}
+	
+	if( wcscmp(key, L"LANG" )==0 )
+	{
+		fish_setlocale(LC_ALL,val);
+	}
+	
+	if( var_mode & ENV_UNIVERSAL )
+	{
+		env_universal_set( key, val );
+		return;
+	}
+	
+	if( val == 0 )
+	{
+		wchar_t *prev_val;
+		free_val = 1;
+		prev_val = env_get( key );
+		val = wcsdup( prev_val?prev_val:L"" );
+	}
+
+	node = env_get_node( key );
+	if( &node->env != 0 )
+	{
+		e = (var_entry_t *) hash_get( &node->env, 
+									  key );
+		
+		if( e->export )
+			has_changed_new = 1;
+		
+	}
+
+	if( (var_mode & ENV_LOCAL) || 
+		(var_mode & ENV_GLOBAL) )
+	{
+		node = ( var_mode & ENV_GLOBAL )?global_env:top;
+	}
+	else
+	{
+		if( node )
+		{
+			if( !(var_mode & ENV_EXPORT ) &&
+				!(var_mode & ENV_UNEXPORT ) )
+			{				
+				var_mode = e->export?ENV_EXPORT:0;
+			}
+		}
+		else
+		{
+			if( env_universal_get( key ) )
+			{
+				env_universal_set( key, val );
+				return;
+			}
+			else
+			{
+				node = top;
+			}
+		}
+	}
+	
+//	env_remove( key, 0 );
+	void *k, *v;
+	hash_remove( &node->env, key, (const void **)&k, (const void **)&v );
+	free( k );
+	free( v );
+
+	entry = malloc( sizeof( var_entry_t ) + 
+					sizeof(wchar_t )*(wcslen(val)+1));
+	
+	if( var_mode & ENV_EXPORT)
+	{
+		entry->export = 1;
+		export_count++;
+		has_changed_new = 1;		
+	}
+	else
+		entry->export = 0;
+
+	wcscpy( entry->val, val );
+
+	hash_put( &node->env, wcsdup(key), entry );
+
+	if( entry->export )
+	{
+		node->export=1;
+	}
+
+	if( free_val )
+		free((void *)val);
+
+//	if( has_changed_new && !has_changed_old )
+//		fwprintf( stderr, L"Reexport after setting %ls to %ls\n", key, val );	
+		
+	has_changed = has_changed_old | has_changed_new;
+	
+}
+
+/**
+   Attempt to remove/free the specified key/value pair from the
+   specified hash table.
+*/
+static int try_remove( env_node_t *n,
+					   const wchar_t *key )
+{
+	wchar_t *old_key, *old_val;
+	if( n == 0 )
+		return 0;
+
+	hash_remove( &n->env, 
+				 key,
+				 (const void **)&old_key, 
+				 (const void **)&old_val );
+	if( old_key != 0 )
+	{
+		var_entry_t * v = (var_entry_t *)old_val;
+		if( v->export )
+		{
+			export_count --;
+			has_changed = 1;
+		}
+		
+		free(old_key);
+		free(old_val);
+		return 1;
+	}
+
+	if( n->new_scope )
+		return try_remove( global_env, key );
+	else
+		return try_remove( n->next, key );
+}
+
+
+void env_remove( const wchar_t *key, int var_mode )
+{
+	if( (var_mode & ENV_USER ) && 
+		hash_get( &env_read_only, key ) )
+	{
+		return;
+	}
+	
+	if( !try_remove( top, key ) )
+	{
+		env_universal_remove( key );
+	}
+}
+
+
+wchar_t *env_get( const wchar_t *key )
+{
+	var_entry_t *res;
+	env_node_t *env = top;
+
+	if( wcscmp( key, L"history" ) == 0 )
+	{
+		wchar_t *current;
+		int i;		
+		int add_current=0;
+		sb_clear( &dyn_var );						
+		
+		current = reader_get_buffer();
+		if( current && wcslen( current ) )
+		{
+			add_current=1;
+			sb_append( &dyn_var, current );
+		}
+		
+		for( i=add_current; i<8; i++ )
+		{
+			wchar_t *next = history_get( i-add_current );
+			if( !next )
+			{
+				debug( 1, L"No history at idx %d\n", i );
+				break;
+			}
+			
+			if( i!=0)
+				sb_append( &dyn_var, ARRAY_SEP_STR );
+			sb_append( &dyn_var, next );
+		}
+		return (wchar_t *)dyn_var.buff;
+	}
+	
+	while( env != 0 )
+	{
+		res = (var_entry_t *) hash_get( &env->env, 
+								 key );
+		if( res != 0 )
+		{
+			return res->val;
+		}
+		
+		if( env->new_scope )
+			env = global_env;
+		else
+			env = env->next;
+	}	
+	return env_universal_get( key );
+}
+
+static int local_scope_exports( env_node_t *n )
+{
+	
+	if( n==global_env )
+		return 0;
+	
+	if( n->export )
+		return 1;
+	
+	if( n->new_scope )
+		return 0;
+	
+	return local_scope_exports( n->next );
+}
+
+void env_push( int new_scope )
+{
+	env_node_t *node = malloc( sizeof(env_node_t) );
+	node->next = top;
+	node->export=0;
+	hash_init( &node->env, &hash_wcs_func, &hash_wcs_cmp );
+	node->new_scope=new_scope;
+	if( new_scope )
+	{
+		has_changed = local_scope_exports(top);
+	}
+	top = node;	
+
+}
+
+/*static int scope_count( env_node_t *n )
+{
+	if( n == global_env )
+		return 0;
+	return( scope_count( n->next) + 1 );
+}
+*/
+
+void env_pop()
+{
+	if( &top->env != global )
+	{
+		env_node_t *killme = top;
+
+		if( killme->new_scope )
+		{
+			has_changed = killme->export || local_scope_exports( killme->next );
+		}
+		
+		top = top->next;
+		hash_foreach( &killme->env, &clear_hash_entry );
+		hash_destroy( &killme->env );
+		free( killme );
+		
+	}
+	else
+	{
+		debug( 0,
+			   L"Tried to pop empty environment stack." );
+		sanity_lose();
+	}	
+}
+
+/**
+   Recreate the table of global variables used by execv
+*/
+static void fill_arr( const void *key, const void *val, void *aux )
+{
+	var_entry_t *val_entry = (var_entry_t *)val;
+	if( val_entry->export )
+	{
+
+		c1++;
+		
+		wchar_t *wcs_val = wcsdup( val_entry->val );
+		wchar_t *pos = wcs_val;
+				
+		int *idx_ptr = (int *)aux;
+		char *key_str = wcs2str((wchar_t *)key);
+		
+		char *val_str;
+		char *woot;
+
+		while( *pos )
+		{
+			if( *pos == ARRAY_SEP )
+				*pos = L':';			
+			pos++;
+		}
+
+		val_str = wcs2str( wcs_val );
+		free( wcs_val );
+		
+		woot = malloc( sizeof(char)*( strlen(key_str) + 
+									  strlen(val_str) + 2) );
+		
+		strcpy( woot, key_str );
+		strcat( woot, "=" );
+		strcat( woot, val_str );
+		export_arr[*idx_ptr] = woot;
+		(*idx_ptr)++;
+		
+		free( key_str );
+		free( val_str );	
+	}
+}
+
+
+/**
+   Function used with hash_foreach to insert keys of one table into
+   another
+*/
+static void add_key_to_hash( const void *key, 
+							 const void *data,
+							 void *aux )
+{
+	var_entry_t *e = (var_entry_t *)data;
+	if( ( e->export && get_names_show_exported) || 
+		( !e->export && get_names_show_unexported) )
+		hash_put( (hash_table_t *)aux, key, 0 );
+}
+static void add_universal_key_to_hash( const void *key, 
+									   const void *data,
+									   void *aux )
+{
+	hash_put( (hash_table_t *)aux, key, 0 );
+}
+
+void env_get_names( array_list_t *l, int flags )
+{
+	int show_local = flags & ENV_LOCAL;
+	int show_global = flags & ENV_GLOBAL;
+	int show_universal = flags & ENV_UNIVERSAL;
+
+	hash_table_t names;
+	env_node_t *n=top;
+
+	get_names_show_exported = 
+		flags & ENV_EXPORT|| (!(flags & ENV_UNEXPORT));
+	get_names_show_unexported = 
+		flags & ENV_UNEXPORT|| (!(flags & ENV_EXPORT));
+
+	if( !show_local && !show_global && !show_universal )
+	{
+		show_local =show_universal = show_global=1;
+	}
+
+	hash_init( &names, &hash_wcs_func, &hash_wcs_cmp );
+	
+	if( show_local )
+	{
+		while( n )
+		{
+			if( n == global_env )
+				break;
+			
+			hash_foreach2( &n->env, 
+						   add_key_to_hash,
+						   &names );
+
+			if( n->new_scope )
+				break;		
+			else
+				n = n->next;
+
+		}
+	}
+	
+	if( show_global )
+	{
+		hash_foreach2( &global_env->env, 
+					   add_key_to_hash,
+					   &names );
+		if( get_names_show_unexported )
+			al_push( l, L"history" );
+	}
+	
+	if( show_universal )
+	{
+		if( get_names_show_unexported )
+			hash_foreach2( &env_universal_var, 
+						   add_universal_key_to_hash,
+						   &names );
+	}
+	
+	hash_get_keys( &names, l );
+	hash_destroy( &names );	
+}
+
+
+char **env_export_arr()
+{
+	if( has_changed )
+	{
+		int pos=0;		
+		char **ptr;		
+		env_node_t *n=top;
+		
+		if( export_arr != 0 )
+		{
+			for( ptr = export_arr; *ptr; ptr++ )
+				free( *ptr );
+		}		
+
+		export_arr = realloc( export_arr,
+							  sizeof(char *)*(export_count + 1) );
+		
+		while( n )
+		{
+			hash_foreach2( &n->env, &fill_arr, &pos );
+			
+			if( n->new_scope )
+				n = global_env;
+			else
+				n = n->next;
+
+		}		
+		export_arr[pos]=0;
+		has_changed=0;
+	}
+	return export_arr;	
+}

env.h

@@ -0,0 +1,98 @@
+/** \file env.h
+	Prototypes for functions for setting and getting environment variables.
+*/
+
+/**
+   Flag for local (to the current block) variable
+*/
+#define ENV_LOCAL 1
+
+/**
+   Flag for exported (to commands) variable
+*/
+#define ENV_EXPORT 2
+
+/**
+   Flag for unexported variable
+*/
+#define ENV_UNEXPORT 16
+
+/**
+   Flag for global variable
+*/
+#define ENV_GLOBAL 4
+
+/**
+   Flag for variable update request from the user. All variable
+   changes that are made directly by the user, such as those from the
+   'set' builtin must have this flag set.
+*/
+#define ENV_USER 8
+
+/**
+   Flag for universal variable
+*/
+#define ENV_UNIVERSAL 32
+
+/**
+   Initialize environment variable data
+*/
+void env_init();
+
+/**
+   Destroy environment variable data
+*/
+void env_destroy();
+
+
+/**
+   Set the value of the environment variable whose name matches key to val. 
+
+   Memory policy: All keys and values are copied, the parameters can and should be freed by the caller afterwards
+
+   \param key The key
+   \param val The value
+   \param mode The type of the variable. Can be any combination of ENV_GLOBAL, ENV_LOCAL, ENV_EXPORT and ENV_USER. If mode is zero, the current variable space is searched and the current mode is used. If no current variable with the same name is found, ENV_LOCAL is assumed.
+
+*/
+
+void env_set( const wchar_t *key, 
+			  const wchar_t *val,
+			  int mode );
+
+
+/**
+  Return the value of the variable with the specified name.
+  Returns 0 if the key does not exist.
+  The returned string should not be modified or freed.
+*/
+wchar_t *env_get( const wchar_t *key );
+
+/**
+   Remove environemnt variable
+   
+   \param key The name of the variable to remove
+   \param mode should be ENV_USER if this is a remove request from the user, 0 otherwise. If this is a user request, read-only variables can not be removed.
+*/
+void env_remove( const wchar_t *key, int mode );
+
+/**
+  Push the variable stack. Used for implementing local variables for functions and for-loops.
+*/
+void env_push( int new_scope );
+
+/**
+  Pop the variable stack. Used for implementing local variables for functions and for-loops.
+*/
+void env_pop();
+
+/**
+   Returns an array containing all exported variables in a format suitable for execv.
+*/
+char **env_export_arr();
+
+/**
+  Insert all variable names into l. These are not copies of the strings and should not be freed after use.
+
+*/
+void env_get_names( array_list_t *l, int flags );

env_universal.c

@@ -0,0 +1,316 @@
+#include <stdlib.h>
+#include <stdio.h>
+#include <wchar.h>
+#include <strings.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/un.h>
+#include <pwd.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <term.h>
+#include <signal.h>
+
+#include "util.h"
+#include "common.h"
+#include "wutil.h"
+#include "env_universal_common.h"
+#include "env_universal.h"
+
+/**
+   Maximum number of times to try to get a new fishd socket
+*/
+
+#define RECONNECT_COUNT 32
+
+
+connection_t env_universal_server;
+
+/**
+   Set to 1 after initialization has been performed
+*/
+static int init = 0;
+
+/**
+   The number of attempts to start fishd
+*/
+static int get_socket_count = 0;
+
+wchar_t * path;
+wchar_t *user;
+void (*start_fishd)();
+
+static int barrier_reply = 0;
+
+static void barrier();
+
+
+/**
+   Get a socket for reading from the server
+*/
+static int get_socket( int fork_ok )
+{
+	int s, len;
+	struct sockaddr_un local;
+	
+	char *name;
+	wchar_t *wdir;
+	wchar_t *wuname;	
+	char *dir =0, *uname=0;
+
+	get_socket_count++;
+	wdir = path;
+	wuname = user;
+	
+	if ((s = socket(AF_UNIX, SOCK_STREAM, 0)) == -1) 
+	{
+		wperror(L"socket");
+		return -1;
+	}
+	
+	if( wdir )
+		dir = wcs2str(wdir );
+	else
+		dir = strdup("/tmp");
+	
+	if( wuname )
+		uname = wcs2str(wuname );
+	else
+	{
+		struct passwd *pw;
+		pw = getpwuid( getuid() );
+		uname = strdup( pw->pw_name );
+	}
+	
+	name = malloc( strlen(dir) +
+				   strlen(uname) + 
+				   strlen(SOCK_FILENAME) + 
+				   2 );
+	
+	strcpy( name, dir );
+	strcat( name, "/" );
+	strcat( name, SOCK_FILENAME );
+	strcat( name, uname );
+	
+	free( dir );
+	free( uname );
+	
+	debug( 3, L"Connect to socket %s at fd %2", name, s );
+	
+	local.sun_family = AF_UNIX;
+	strcpy(local.sun_path, name );
+	free( name );
+	len = strlen(local.sun_path) + sizeof(local.sun_family);
+	
+	if( connect( s, (struct sockaddr *)&local, len) == -1 ) 
+	{
+		close( s );
+		if( fork_ok )
+		{
+			debug( 2, L"Could not connect to socket %d, starting fishd", s );
+			if( start_fishd )
+				start_fishd();
+			return get_socket( 0 );
+		}
+		
+		debug( 3, L"Could not connect to socket %d, already tried forking, giving up", s );
+		return -1;
+	}
+	
+	if( fcntl( s, F_SETFL, O_NONBLOCK ) != 0 )
+	{
+		wperror( L"fcntl" );
+		close( s );		
+		
+		return -1;
+	}
+	
+	debug( 3, L"Connected to fd %d", s );
+	
+	return s;
+}
+
+static void callback( int type, const wchar_t *name, const wchar_t *val )
+{
+	if( type == BARRIER_REPLY )
+	{
+		debug( 3, L"Got barrier reply" );
+		
+		barrier_reply = 1;
+	}
+	
+}
+
+
+void env_universal_init( wchar_t * p, wchar_t *u, void (*sf)() )
+{
+	debug( 2, L"env_universal_init()" );
+	path=p;
+	user=u;
+	start_fishd=sf;	
+	env_universal_server.fd = -1;
+	env_universal_server.killme = 0;
+	env_universal_server.fd = get_socket(1);
+	memset (&env_universal_server.wstate, '\0', sizeof (mbstate_t));
+	q_init( &env_universal_server.unsent );
+	env_universal_common_init( &callback );
+	sb_init( &env_universal_server.input );	
+	env_universal_read_all();	
+	init = 1;	
+	if( env_universal_server.fd >= 0 )
+	{
+		barrier();
+	}
+	debug( 2, L"end env_universal_init()" );
+}
+
+void env_universal_destroy()
+{
+	/*
+	  Go into blocking mode and send all data before exiting
+	*/
+	if( env_universal_server.fd >= 0 )
+	{
+		if( fcntl( env_universal_server.fd, F_SETFL, 0 ) != 0 )
+		{
+			wperror( L"fcntl" );
+		}
+		try_send_all( &env_universal_server );	
+	}
+	close( env_universal_server.fd );
+	env_universal_server.fd =-1;
+	q_destroy( &env_universal_server.unsent );
+	sb_destroy( &env_universal_server.input );	
+	env_universal_common_destroy();
+	init = 0;
+}
+
+
+/**
+   Read all available messages from the server.
+*/
+int env_universal_read_all()
+{
+	if( !init)
+		return 0;
+
+	if( env_universal_server.fd == -1 )
+	{
+		
+		if( get_socket_count >= RECONNECT_COUNT )
+			return 0;
+
+		debug( 2, L"Get new fishd connection" );
+
+		init = 0;
+		env_universal_server.fd = get_socket(1);
+		init = 1;
+
+		if( env_universal_server.fd >= 0 )
+			barrier();
+	}
+	
+	if( env_universal_server.fd != -1 )
+	{
+		read_message( &env_universal_server );
+		if( env_universal_server.killme )
+		{
+			debug( 2, L"Lost connection to universal variable server." );
+			close( env_universal_server.fd );
+			env_universal_server.fd = -1;
+			env_universal_server.killme=0;
+			sb_clear( &env_universal_server.input );	
+
+			env_universal_read_all();
+		}
+		return 1;
+	}
+	else
+	{
+		debug( 2, L"No connection to universal variable server" );
+		return 0;
+	}		
+}
+
+wchar_t *env_universal_get( const wchar_t *name )
+{
+	if( !init)
+		return 0;
+	
+	debug( 2, L"env_universal_get( %ls )", name );
+	barrier();
+	
+	if( !name )
+		return 0;
+	
+	return (wchar_t *)hash_get( &env_universal_var, name );	
+}
+
+static void barrier()
+{
+	message_t *msg;
+	fd_set fds;
+	
+	barrier_reply = 0;
+
+	msg= create_message( BARRIER, 0, 0);
+	msg->count=1;
+	q_put( &env_universal_server.unsent, msg );
+
+	debug( 3, L"Create barrier" );
+	while( 1 )
+	{
+		try_send_all( &env_universal_server );	
+		if( q_empty( &env_universal_server.unsent ) )
+			break;
+		FD_ZERO( &fds );
+		FD_SET( env_universal_server.fd, &fds );
+		select( env_universal_server.fd+1, 0, &fds, 0, 0 );
+	}
+
+	debug( 3, L"Sent barrier request" );
+	while( !barrier_reply )
+	{
+		FD_ZERO( &fds );
+        FD_SET( env_universal_server.fd, &fds );		
+        select( env_universal_server.fd+1, &fds, 0, 0, 0 );
+		env_universal_read_all();
+	}
+	debug( 3, L"End barrier" );
+
+}
+
+
+void env_universal_set( const wchar_t *name, const wchar_t *value )
+{
+	message_t *msg;
+	
+	if( !init )
+		return;
+	
+	debug( 2, L"env_universal_set( %ls, %ls )", name, value );
+
+	msg= create_message( SET, name, value);
+	msg->count=1;
+	q_put( &env_universal_server.unsent, msg );
+	barrier();
+	
+
+}
+
+void env_universal_remove( const wchar_t *name )
+{
+	message_t *msg;
+	if( !init )
+		return;
+	
+	debug( 2,
+		   L"env_universal_remove( %ls )",
+		   name );
+
+	msg= create_message( ERASE, name, 0);
+	msg->count=1;
+	q_put( &env_universal_server.unsent, msg );
+	barrier();
+}

env_universal.h

@@ -0,0 +1,39 @@
+/** \file env_universl.h
+	Universal variable client library
+*/
+
+#ifndef ENV_UNIVERSAL_HH
+#define ENV_UNIVERSAL_HH
+
+#include "env_universal_common.h"
+
+/**
+   Data about the universal variable server.
+*/
+extern connection_t env_universal_server;
+
+/**
+   Initialize the envuni library
+*/
+void env_universal_init();
+/*
+  Free memory used by envuni
+*/
+void env_universal_destroy();
+
+/**
+   Get the value of a universal variable
+*/
+wchar_t *env_universal_get( const wchar_t *name );
+/**
+   Set the value of a universal variable
+*/
+void env_universal_set( const wchar_t *name, const wchar_t *val );
+/**
+   Erase a universal variable
+*/
+void env_universal_remove( const wchar_t *name );
+
+int env_universal_read_all();
+
+#endif

env_universal_common.c

@@ -0,0 +1,403 @@
+/**
+   \file env_universal_common.c
+
+   The utility library for universal variables. Used both by the
+   client library and by the daemon.
+
+*/
+#include <stdlib.h>
+#include <stdio.h>
+#include <wchar.h>
+#include <strings.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/un.h>
+#include <pwd.h>
+#include <errno.h>
+#include <sys/stat.h>
+#include <dirent.h>
+#include <wctype.h>
+
+#include <errno.h>
+#include <locale.h>
+#include <dirent.h>
+#include <signal.h>
+#include <sys/stat.h>
+
+#include "util.h"
+#include "common.h"
+#include "wutil.h"
+#include "env_universal_common.h"
+
+/**
+   Non-wide version of the set command
+*/
+#define SET_MBS "SET"
+
+/**
+   Non-wide version of the erase command
+*/
+#define ERASE_MBS "ERASE"
+
+#define BARRIER_MBS "BARRIER"
+#define BARRIER_REPLY_MBS "BARRIER_REPLY"
+
+/**
+   Error message
+*/
+#define PARSE_ERR L"Unable to parse universal variable message: '%ls'"
+
+static void parse_message( wchar_t *msg, connection_t *src );
+
+/**
+   The table of all universal variables
+*/
+hash_table_t env_universal_var;
+
+void (*callback)(int type, const wchar_t *key, const wchar_t *val );
+
+
+
+void env_universal_common_init(void (*cb)(int type, const wchar_t *key, const wchar_t *val ) )
+{
+	debug( 2, L"Init env_universal_common" );
+	callback = cb;
+	hash_init( &env_universal_var, &hash_wcs_func, &hash_wcs_cmp );
+}
+
+static void erase( const void *key,
+				   const void *data )
+{
+	free( (void *)key );
+	free( (void *)data );
+}
+
+
+void env_universal_common_destroy()
+{
+	hash_foreach( &env_universal_var, &erase );
+	hash_destroy( &env_universal_var );
+}
+
+
+void read_message( connection_t *src )
+{
+	while( 1 )
+	{
+		char b;		
+		int read_res = read( src->fd, &b, 1 );
+		wchar_t res=0;
+		
+		if( read_res < 0 )
+		{
+			if( errno != EAGAIN && 
+				errno != EINTR )
+			{
+				debug( 2, L"Read error on fd %d, set killme flag", src->fd );
+				wperror( L"read" );
+				src->killme = 1;
+			}
+			return;
+		}
+		if( read_res == 0 )
+		{
+			src->killme = 1;
+			debug( 3, L"Fd %d has reached eof, set killme flag", src->fd );
+			if( src->input.used > 0 )
+			{
+				debug( 1, 
+					   L"Universal variable connection closed while reading command. Partial command recieved: '%ls'", 
+					   (wchar_t *)src->input.buff  );
+			}
+			return;
+		}
+		
+		int sz = mbrtowc( &res, &b, 1, &src->wstate );
+		
+		if( sz == -1 )
+		{			
+			debug( 1, L"Error while reading universal variable after '%ls'", (wchar_t *)src->input.buff  );
+			wperror( L"mbrtowc" );
+		}
+		else if( sz > 0 )
+		{
+			if( res == L'\n' )
+			{
+				parse_message( (wchar_t *)src->input.buff, src );	
+				sb_clear( &src->input );
+				memset (&src->wstate, '\0', sizeof (mbstate_t));
+			}
+			else
+			{
+				sb_printf( &src->input, L"%lc", res );
+			}
+		}
+	}
+}
+
+static void remove_entry( wchar_t *name )
+{
+	void *k, *v;
+	hash_remove( &env_universal_var, 
+				 name,
+				 (const void **)&k,
+				 (const void **)&v );
+	free( k );
+	free( v );
+}
+
+static int match( const wchar_t *msg, const wchar_t *cmd )
+{
+	size_t len = wcslen( cmd );
+	if( wcsncasecmp( msg, cmd, len ) != 0 )
+		return 0;
+	if( msg[len] && msg[len]!= L' ' && msg[len] != L'\t' )
+		return 0;
+	
+	return 1;
+}
+
+
+static void parse_message( wchar_t *msg, connection_t *src )
+{
+	debug( 2, L"parse_message( %ls );", msg );
+
+	if( msg[0] == L'#' )
+		return;
+		
+	if( match( msg, SET_STR ) )
+	{
+		wchar_t *name, *val, *tmp;
+			
+		name = msg+wcslen(SET_STR);
+		while( wcschr( L"\t ", *name ) )
+			name++;
+		
+		tmp = wcschr( name, L':' );
+		if( tmp )
+		{
+			wchar_t *key =malloc( sizeof( wchar_t)*(tmp-name+1));
+			memcpy( key, name, sizeof( wchar_t)*(tmp-name));
+			key[tmp-name]=0;
+			
+			val = tmp+1;
+			
+			val = unescape( wcsdup(val), 0 );
+			
+			remove_entry( key );
+			
+			hash_put( &env_universal_var, key, val );
+			
+			if( callback )
+			{
+				callback( SET, key, val );
+			}
+		}
+		else
+		{
+			debug( 1, PARSE_ERR, msg );
+		}			
+	}
+	else if( match( msg, ERASE_STR ) )
+	{
+		wchar_t *name, *val, *tmp;
+		
+		name = msg+wcslen(ERASE_STR);
+		while( wcschr( L"\t ", *name ) )
+			name++;
+		
+		tmp = name;
+		while( iswalnum( *tmp ) || *tmp == L'_')
+			tmp++;
+		
+		*tmp = 0;
+		
+		if( !wcslen( name ) )
+		{
+			debug( 1, PARSE_ERR, msg );
+		}
+
+		remove_entry( name );
+		
+		if( callback )
+		{
+			callback( ERASE, name, 0 );
+		}
+	}
+	else if( match( msg, BARRIER_STR) )
+	{
+		message_t *msg = create_message( BARRIER_REPLY, 0, 0 );
+		msg->count = 1;
+		q_put( &src->unsent, msg );
+		try_send_all( src );
+	}
+	else if( match( msg, BARRIER_REPLY_STR ) )
+	{
+		if( callback )
+		{
+			callback( BARRIER_REPLY, 0, 0 );
+		}
+	}
+	else
+	{
+		debug( 1, PARSE_ERR, msg );
+	}		
+}
+
+int try_send( message_t *msg, int fd )
+{
+
+	int res = write( fd, msg->body, strlen(msg->body) );
+	
+	if( res == -1 )
+	{
+		switch( errno )
+		{
+			case EAGAIN:
+				return 0;
+				
+			default:
+				debug( 1,
+					   L"Error while sending message to fd %d. Closing connection",
+					   fd );
+				wperror( L"write" );
+				
+				return -1;
+		}		
+	}
+	msg->count--;
+	
+	if( !msg->count )
+	{
+		free( msg );
+	}
+	return 1;
+}
+
+void try_send_all( connection_t *c )
+{
+	debug( 2,
+		   L"Send all updates to connection on fd %d", 
+		   c->fd );
+	while( !q_empty( &c->unsent) )
+	{
+		
+		switch( try_send( (message_t *)q_peek( &c->unsent), c->fd ) )
+		{
+			case 1:
+				q_get( &c->unsent);
+				break;
+				
+			case 0:
+				return;
+								
+			case -1:
+				c->killme = 1;
+				return;
+		}
+	}
+}
+
+message_t *create_message( int type,
+						   const wchar_t *key_in, 
+						   const wchar_t *val_in )
+{
+	message_t *msg=0;
+	
+	char *key=0;
+	size_t sz;
+	
+	if( key_in )
+	{
+		key = wcs2str(key_in);
+		if( !key )
+		{
+			debug( 0,
+				   L"Could not convert %ls to narrow character string",
+				   key_in );
+			return 0;
+		}
+	}
+	
+	
+	switch( type )
+	{
+		case SET:
+		{
+			if( !val_in )
+			{
+				val_in=L"";
+			}
+			
+			wchar_t *esc = escape(wcsdup(val_in),1);
+			if( !esc )
+				break;
+			
+			char *val = wcs2str(esc );
+			free(esc);
+			
+			
+			sz = strlen(SET_MBS) + strlen(key) + strlen(val) + 4;
+			msg = malloc( sizeof( message_t ) + sz );
+	
+			if( !msg )
+				die_mem();
+				
+			strcpy( msg->body, SET_MBS " " );
+			strcat( msg->body, key );
+			strcat( msg->body, ":" );
+			strcat( msg->body, val );
+			strcat( msg->body, "\n" );
+
+			free( val );
+			
+			break;
+		}
+
+		case ERASE:
+		{
+            sz = strlen(ERASE_MBS) + strlen(key) + 3;
+            msg = malloc( sizeof( message_t ) + sz );
+
+			if( !msg )
+				die_mem();
+			
+            strcpy( msg->body, ERASE_MBS " " );
+            strcat( msg->body, key );
+            strcat( msg->body, "\n" );
+            break;
+		}
+
+		case BARRIER:
+		{
+			msg = malloc( sizeof( message_t )  + 
+						  strlen( BARRIER_MBS ) +2);
+			if( !msg )
+				die_mem();
+			strcpy( msg->body, BARRIER_MBS "\n" );
+			break;
+		}
+		
+		case BARRIER_REPLY:
+		{
+			msg = malloc( sizeof( message_t )  +
+						  strlen( BARRIER_REPLY_MBS ) +2);
+			if( !msg )
+				die_mem();
+			strcpy( msg->body, BARRIER_REPLY_MBS "\n" );
+			break;
+		}
+		
+		default:
+		{
+			debug( 0, L"create_message: Unknown message type" );
+		}
+	}
+
+	free( key );
+
+	if( msg )
+		msg->count=0;
+	return msg;	
+}

env_universal_common.h

@@ -0,0 +1,110 @@
+#ifndef ENV_UNIVERSAL_COMMON_HH
+#define ENV_UNIVERSAL_COMMON_HH
+
+
+/**
+   The set command
+*/
+#define SET_STR L"SET"
+
+/**
+   The erase command
+*/
+#define ERASE_STR L"ERASE"
+
+#define BARRIER_STR L"BARRIER"
+#define BARRIER_REPLY_STR L"BARRIER_REPLY"
+
+
+/**
+   The filename to use for univeral variables. The username is appended
+*/
+#define SOCK_FILENAME "fishd.socket."
+
+/**
+   The different types of commands that can be sent between client/server
+*/
+enum
+{
+	SET,
+	ERASE,
+	BARRIER,
+	BARRIER_REPLY,
+}
+	;
+
+/**
+   The table of universal variables
+*/
+extern hash_table_t env_universal_var;
+
+/**
+   This struct represents a connection between a universal variable server/client
+*/
+typedef struct connection
+{
+	/**
+	   The file descriptor this socket lives on
+	*/
+	int fd;
+	/**
+	   Queue of onsent messages
+	*/
+	queue_t unsent;
+	/**
+	   Set to one when this connection should be killed
+	*/
+	int killme;
+	/**
+	   The state used for character conversions
+	*/
+	mbstate_t wstate;
+	/**
+	   The input string. Input from the socket goes here. When a
+	   newline is encountered, the buffer is parsed and cleared.
+	*/
+	string_buffer_t input;
+	
+	/**
+	   Link to the next connection
+	*/
+	struct connection *next;
+}
+	connection_t;
+
+/**
+   A struct representing a message to be sent between client and server
+*/
+typedef struct 
+{
+	int count;
+	char body[0];
+}
+	message_t;
+
+/**
+   Read all available messages on this connection
+*/
+void read_message( connection_t * );
+
+/**
+   Send as many messages as possible without blocking to the connection
+*/
+void try_send_all( connection_t *c );
+
+/**
+   Create a messge with the specified properties
+*/
+message_t *create_message( int type, const wchar_t *key, const wchar_t *val );
+
+/**
+   Init the library
+*/
+void env_universal_common_init(void (*cb)(int type, const wchar_t *key, const wchar_t *val ) );
+
+/**
+   Destroy library data
+*/
+void env_universal_common_destroy();
+
+#endif

exec.h

@@ -0,0 +1,43 @@
+/** \file exec.h
+	Prototypes for functions for executing a program
+*/
+
+/**
+  Execute the processes specified by j. 
+
+   I've put a fair bit of work into making builtins behave like other
+   programs as far as pipes are concerned. Unlike i.e. bash, builtins
+   can pipe to other builtins with arbitrary amounts of data, and so
+   on. To do this, after a builtin is run in the real process, it
+   forks and a dummy process is created, responsible for writing the
+   output of the builtin. This is surprisingly cheap on my computer,
+   probably because of the marvels of copy on write forking.
+
+   This rule is short circuted in the case where a builtin does not
+   output to a pipe and does in fact not output anything. The speed
+   improvement from this optimization is not noticable on a normal
+   computer/OS in regular use, but the promiscous amounts of forking
+   that resulted was responsible for a huge slowdown when using
+   Valgrind as well as when doing complex command-specific
+   completions.
+
+
+*/
+void exec( job_t *j );
+
+/**
+  Evaluate the expression cmd in a subshell, add the outputs into the
+  list l. On return, the status flag as returned bu \c
+  proc_gfet_last_status will not be changed.
+
+  \param cmd the command to execute
+  \param l The list to insert output into.If \c l is zero, the output will be discarded.
+
+  \return the status of the last job to exit, or -1 if en error was encountered.
+*/
+int exec_subshell( const wchar_t *cmd, 
+				   array_list_t *l );
+
+void exec_free_io_buffer( io_data_t *io_buffer );
+io_data_t *exec_make_io_buffer();
+

expand.h

@@ -0,0 +1,170 @@
+/**\file expand.h
+
+   Prototypes for string expantion functions. These functions perform
+   several kinds of parameter expantion. There are a lot of issues
+   with regards to memory allocation. Overall, these functions would
+   benefit from using a more clever memory allocation scheme, perhaps
+   an evil combination of talloc, string buffers and reference
+   counting.
+
+*/
+
+/**
+   Flag specifying that subshell expantion should be skipped
+*/
+#define EXPAND_SKIP_SUBSHELL 1
+
+/**
+   Flag specifying that variable expantion should be skipped
+*/
+#define EXPAND_SKIP_VARIABLES 2
+
+/**
+   Flag specifying that wildcard expantion should be skipped
+*/
+#define EXPAND_SKIP_WILDCARDS 4
+
+/**
+   Incomplete matches in the last segment are ok (for tab
+   completion). An incomplete match is a wildcard that matches a
+   prefix of the filename. If accept_incomplete is true, only the
+   remainder of the string is returned.
+*/
+#define ACCEPT_INCOMPLETE 8
+
+/**
+   Only match files that are executable by the current user. Only applicable together with ACCEPT_INCOMPLETE.
+*/
+
+#define EXECUTABLES_ONLY 16
+
+/**
+   Only match directories. Only applicable together with ACCEPT_INCOMPLETE.
+*/
+
+#define DIRECTORIES_ONLY 32
+
+/** Character represeting a home directory */
+#define HOME_DIRECTORY 0xfffffffc
+
+/** Character represeting process expantion */
+#define PROCESS_EXPAND 0xfffffffb
+
+/** Character representing variable expantion */
+#define VARIABLE_EXPAND 0xfffffffa
+
+/** Character representing the start of a bracket expantion */
+#define BRACKET_BEGIN 0xfffffff9
+
+/** Character representing the end of a bracket expantion */
+#define BRACKET_END 0xfffffff8
+
+/** Character representing separation between two bracket elements */
+#define BRACKET_SEP 0xfffffff7
+
+/** Character for separating two array elements. We use 30, i.e. the ascii record separator since that seems logical. */
+#define ARRAY_SEP 0x1e
+
+/** String containing the character for separating two array elements */
+#define ARRAY_SEP_STR L"\x1e"
+
+/**
+   Separate subtokens in a token with this character. 
+*/
+#define INTERNAL_SEPARATOR 0xfffffff0
+
+
+
+/**
+   Perform various forms of expansion on in, such as tilde expansion
+   (~USER becomes the users home directory), variable expansion
+   ($VAR_NAME becomes the value of the environment variable VAR_NAME),
+   subshell expantion and wildcard expansion. The results are inserted
+   into the list out.
+   
+   If the parameter does not need expansion, it is copied into the list
+   out. If expansion is performed, the original parameter is freed and
+   newly allocated strings are inserted into the list out.
+   
+   \param in The parameter to expand
+   \param flag Specifies if any expantion pass should be skipped. Legal values are any combination of EXPAND_SKIP_SUBSHELL EXPAND_SKIP_VARIABLES and EXPAND_SKIP_WILDCARDS
+   \param out The list to which the result will be appended.
+*/
+int expand_string( wchar_t *in, array_list_t *out, int flag );
+
+/**
+   expand_one is identical to expand_string, except it will fail if in
+   expands to more than one string. This is used for expanding command
+   names.
+
+   \param in The parameter to expand
+   \param flag Specifies if any expantion pass should be skipped. Legal values are any combination of EXPAND_SKIP_SUBSHELL EXPAND_SKIP_VARIABLES and EXPAND_SKIP_WILDCARDS
+   \return The expanded parameter, or 0 on failiure
+*/
+wchar_t *expand_one( wchar_t *in, int flag );
+
+/**
+   Replace special characters with escape sequences. Newline is
+   replaced with \n, etc. 
+
+   \param in The string to be escaped
+   \param escape_all Whether all characters wich hold special meaning in fish (Pipe, semicolon, etc,) should be escaped, or only unprintable characters
+   \return The escaped string, or 0 if there is not enough memory
+*/
+wchar_t *expand_escape( wchar_t *in, int escape_all );
+
+/**
+   Convert the variable value to a human readable form, i.e. escape things, handle arrays, etc. Suitable for pretty-printing.
+*/
+wchar_t *expand_escape_variable( const wchar_t *in );
+
+
+/**
+   Perform tilde expantion and nothing else on the specified string.
+
+   If tilde expantion is needed, the original string is freed and a
+   new string, allocated using malloc, is returned.
+*/
+wchar_t *expand_tilde(wchar_t *in);
+
+/**
+   Locate the last subshell in the specified string.
+   
+   \param in the string to search for subshells
+   \param begin the starting paranthesis of the subshell
+   \param end the ending paranthesis of the subshell
+   \param flags set this variable to ACCEPT_INCOMPLETE if in tab_completion mode
+   \return -1 on syntax error, 0 if no subshells exist and 1 on sucess
+*/
+int expand_locate_subshell( wchar_t *in, 
+							wchar_t **begin, 
+							wchar_t **end,
+							int flags );
+
+
+/**
+   Expand backslashed escapes and substitute them with their unescaped
+   counterparts. Also optionally change the wildcards, the tilde
+   character and a few more into constants which are defined to be
+   outside of the valid character space, but still inside the valid
+   space for a wchar_t. This assumes that a wchar_t is at least 32
+   bits long AND that the characterset is UCS4 or some other 31-bit
+   character set.
+
+   Since removing the escape sequences can never lengthen the string,
+   the specified string is modified instead of allocating a new one.
+
+*/
+wchar_t *expand_backslash( wchar_t * in, int escape_special );
+
+
+/**
+   Tokenize the specified string into the specified array_list_t.
+   Each new element is allocated using malloc and must be freed by the
+   caller.
+   
+   \param val the input string. The contents of this string is not changed.
+   \param out the list in which to place the elements. 
+*/
+void expand_variable_array( const wchar_t *val, array_list_t *out );
+

fish.spec.in

@@ -0,0 +1,110 @@
+Summary:                A friendly interactive shell
+Name:                   @PACKAGE_NAME@
+
+Version:                @PACKAGE_VERSION@
+Release:                0%{?dist}
+
+License:                GPL
+Group:                  System Environment/Shells
+URL:                    http://roo.no-ip.org/fish/
+
+Source0:                http://roo.no-ip.org/%{name}/files/%{version}/%{name}-%{version}.tar.bz2
+
+BuildRoot:              %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
+BuildRequires:          doxygen ncurses-devel xorg-x11-devel
+
+%description 
+fish is a shell geared towards interactive use. It's features are
+focused on user friendlieness and discoverability. The language syntax
+is simple but incompatible with other shell languages.
+
+%prep
+%setup -q
+
+%build
+
+%configure docdir=%_datadir/doc/%{name}-%{version}
+make %{?_smp_mflags}
+
+%install
+rm -rf $RPM_BUILD_ROOT
+make install DESTDIR="$RPM_BUILD_ROOT" 
+
+%clean
+rm -rf $RPM_BUILD_ROOT
+
+%post
+if ! grep %_bindir/fish %_sysconfdir/shells >/dev/null; then
+	echo %_bindir/fish >>%_sysconfdir/shells
+fi
+
+%postun
+if [ "$1" = 0 ]; then
+	grep -v %_bindir/fish %_sysconfdir/shells >%_sysconfdir/fish.tmp
+	mv %_sysconfdir/fish.tmp %_sysconfdir/shells
+fi
+
+%files
+%defattr(-,root,root,-)
+%doc %_datadir/doc/%{name}-%{version}
+%_mandir/man1/fish.1*
+%_mandir/man1/xsel.1x*
+%_mandir/man1/mimedb.1*
+%_mandir/man1/set_color.1*
+%_mandir/man1/tokenize.1*
+%_mandir/man1/count.1*
+%attr(0755,root,root) %_bindir/fish
+%attr(0755,root,root) %_bindir/fishd
+%attr(0755,root,root) %_bindir/fish_pager
+%attr(0755,root,root) %_bindir/xsel
+%attr(0755,root,root) %_bindir/set_color
+%attr(0755,root,root) %_bindir/tokenize
+%attr(0755,root,root) %_bindir/mimedb
+%attr(0755,root,root) %_bindir/count
+%config %_sysconfdir/fish
+%config %_sysconfdir/fish_inputrc
+%dir %_sysconfdir/fish.d
+%config %_sysconfdir/fish.d/fish_*.fish
+%dir %_sysconfdir/fish.d/completions
+%config %_sysconfdir/fish.d/completions/*.fish
+
+%changelog
+* Mon Sep 12 2005 Axel Liljencrantz <axel@liljencrantz.se> 1.13.4-0
+- 1.13.4
+
+* Wed Sep 07 2005 Axel Liljencrantz <axel@liljencrantz.se> 1.13.3-0
+- 1.13.3
+
+* Tue Sep 06 2005 Axel Liljencrantz <axel@liljencrantz.se> 1.13.2-0
+- 1.13.2
+
+* Fri Aug 30 2005 Axel Liljencrantz <axel@liljencrantz.se> 1.13.1-0
+- 1.13.1
+
+* Sun Aug 28 2005 Axel Liljencrantz <axel@liljencrantz.se> 1.13.0-0
+- 1.13.0
+
+* Sat Aug 13 2005 Axel Liljencrantz <axel@liljencrantz.se> 1.13.0-0
+- Add completions subdirectory
+
+* Thu Jul 28 2005 Axel Liljencrantz <axel@liljencrantz.se> 1.12.1-0
+- 1.12.1
+
+* Fri Jul 15 2005 Axel Liljencrantz <axel@liljencrantz.se> 1.12.0-1
+- 1.12.0
+
+* Thu Jun 30 2005 Michael Schwendt <mschwendt@users.sf.net> 1.11.1-9
+- Set CFLAGS the proper way
+
+* Thu Jun 30 2005 Axel Liljencrantz <axel@liljencrantz.se> 1.11.1-8
+- Fix revision number in changelog
+
+* Wed Jun 29 2005 Axel Liljencrantz <axel@liljencrantz.se> 1.11.1-7
+- Send post-script output to /dev/null
+
+* Wed Jun 29 2005 Axel Liljencrantz <axel@liljencrantz.se> 1.11.1-6
+- Add changelog section to spec file
+- Add macros to source tags
+- Add smp_mflags to 'make all'
+- Fix typo in post install scriptlet test
+- Set CFLAGS from spec file

fish_pager.c

@@ -0,0 +1,992 @@
+#include "config.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <wchar.h>
+#include <unistd.h>
+#include <termios.h>
+#include <string.h>
+
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/ioctl.h>
+#include <sys/time.h>
+#include <sys/wait.h>
+#include <dirent.h>
+#include <fcntl.h>
+
+#include <locale.h>
+
+#if HAVE_NCURSES_H
+#include <ncurses.h>
+#else
+#include <curses.h>
+#endif
+
+#if HAVE_TERMIO_H
+#include <termio.h>
+#endif
+
+#include <term.h>
+#include <signal.h>
+
+#include "util.h"
+#include "wutil.h"
+#include "common.h"
+#include "complete.h"
+#include "output.h"
+#include "input_common.h"
+#include "env_universal.h"
+
+#define WCHAR_END 0x80000000
+
+enum 
+{
+	LINE_UP = R_NULL+1,
+	LINE_DOWN,
+	PAGE_UP,
+	PAGE_DOWN
+}
+	;
+
+
+enum
+{
+	HIGHLIGHT_PAGER_PREFIX,
+	HIGHLIGHT_PAGER_COMPLETION,
+	HIGHLIGHT_PAGER_DESCRIPTION,
+	HIGHLIGHT_PAGER_PROGRESS
+}
+;
+
+/**
+   This struct should be continually updated by signals as the term
+   resizes, and as such always contain the correct current size.
+*/
+
+static struct winsize termsize;
+
+static struct termios saved_modes;
+static struct termios pager_modes;
+
+static int is_ca_mode = 0;
+
+
+/**
+   The environment variables used to specify the color of different
+   tokens.
+*/
+static wchar_t *hightlight_var[] = 
+{
+	L"fish_pager_color_prefix",
+	L"fish_pager_color_completion",
+	L"fish_pager_color_description",
+	L"fish_pager_color_progress"
+}
+	;
+
+static string_buffer_t out_buff;
+static FILE *out_file;
+
+
+int get_color( int highlight )
+{
+	if( highlight < 0 )
+		return FISH_COLOR_NORMAL;
+	if( highlight >= (4) )
+		return FISH_COLOR_NORMAL;
+	
+	wchar_t *val = env_universal_get( hightlight_var[highlight]);
+	
+	if( val == 0 )
+		return FISH_COLOR_NORMAL;
+	
+	if( val == 0 )
+	{
+		return FISH_COLOR_NORMAL;
+	}
+	
+	return output_color_code( val );	
+}
+
+int try_sequence( char *seq )
+{
+	int j, k;
+	wint_t c=0;
+	
+	for( j=0; 
+		 seq[j] != '\0' && seq[j] == (c=input_common_readch( j>0 )); 
+		 j++ )
+		;
+
+	if( seq[j] == '\0' )
+	{		
+		return 1;
+	}
+	else
+	{
+		input_common_unreadch(c);
+		for(k=j-1; k>=0; k--)
+			input_common_unreadch(seq[k]);
+	}
+	return 0;
+}
+
+static wint_t readch()
+{
+	struct mapping
+	{
+		char *seq;
+		wint_t bnd;
+	}
+	;
+	
+	struct mapping m[]=
+		{
+			{				
+				"\e[A", LINE_UP
+			}
+			,
+			{
+				key_up, LINE_UP
+			}
+			,
+			{				
+				"\e[B", LINE_DOWN
+			}
+			,
+			{
+				key_down, LINE_DOWN
+			}
+			,
+			{
+				key_ppage, PAGE_UP
+			}
+			,
+			{
+				key_npage, PAGE_DOWN
+			}
+			,
+			{
+				" ", PAGE_DOWN
+			}
+			,
+			{
+				"\t", PAGE_DOWN
+			}
+			,
+			{
+				0, 0
+			}
+			
+		}
+	;
+	int i;
+	
+	for( i=0; m[i].seq; i++ )
+	{
+		if( try_sequence(m[i].seq ) )
+			return m[i].bnd;
+	}
+	return input_common_readch(0);
+}
+
+
+/**
+   Print the specified part of the completion list, using the
+   specified column offsets and quoting style.
+
+   \param l The list of completions to print
+   \param cols number of columns to print in
+   \param width An array specifying the width of each column
+   \param row_start The first row to print
+   \param row_stop the row after the last row to print
+   \param prefix The string to print before each completion
+   \param is_quoted Whether to print the completions are in a quoted environment
+*/
+
+static void completion_print( int cols,
+							  int *width,
+							  int row_start,
+							  int row_stop,
+							  wchar_t *prefix,
+							  int is_quoted,
+							  array_list_t *l)
+{
+
+	int rows = (al_get_count( l )-1)/cols+1;
+	int i, j;
+	int prefix_width= my_wcswidth(prefix);
+
+	for( i = row_start; i<row_stop; i++ )
+	{
+		for( j = 0; j < cols; j++ )
+		{
+			wchar_t *el, *el_end;
+
+			if( al_get_count( l ) <= j*rows + i )
+				continue;
+
+			el = (wchar_t *)al_get( l, j*rows + i );
+			el_end= wcschr( el, COMPLETE_SEP );
+
+			set_color( get_color(HIGHLIGHT_PAGER_PREFIX),FISH_COLOR_NORMAL );
+
+			writestr( prefix );
+
+			set_color( get_color(HIGHLIGHT_PAGER_COMPLETION),FISH_COLOR_IGNORE );
+
+			if( el_end == 0 )
+			{
+				/* We do not have a description for this completion */
+				int written = 0;
+				int max_written = width[j] - prefix_width - (j==cols-1?0:2);
+
+				if( is_quoted )
+				{
+					for( i=0; i<max_written; i++ )
+					{
+						if( !el[i] )
+							break;
+						writech( el[i] );
+						written+= wcwidth( el[i] );
+					}
+				}
+				else
+				{
+					written = write_escaped_str( el, max_written );
+				}
+
+				set_color( get_color( HIGHLIGHT_PAGER_DESCRIPTION ),
+						   FISH_COLOR_IGNORE );
+
+				writespace( width[j]-
+							written-
+							prefix_width );
+			}
+			else
+			{
+				int whole_desc_width = my_wcswidth(el_end+1);
+				int whole_comp_width;
+
+				/*
+				  Temporarily drop the description so that wcswidth et
+				  al only calculate the width of the completion.
+				*/
+				*el_end = L'\0';
+
+				/*
+				  Calculate preferred completion width
+				*/
+				if( is_quoted )
+				{
+					whole_comp_width = my_wcswidth(el);
+				}
+				else
+				{
+					wchar_t *tmp = escape( wcsdup(el), 1 );
+					whole_comp_width = my_wcswidth( tmp );
+					free(tmp);
+				}
+
+				/*
+				  Calculate how wide this entry 'wants' to be
+				*/
+				int pref_width = whole_desc_width + 4 + prefix_width + 2 -
+					(j==cols-1?2:0) + whole_comp_width;
+
+				int comp_width, desc_width;
+
+				if( pref_width <= width[j] )
+				{
+					/*
+					  The entry fits, we give it as much space as it wants
+					*/
+					comp_width = whole_comp_width;
+					desc_width = whole_desc_width;
+				}
+				else
+				{
+					/*
+					  The completion and description won't fit on the
+					  allocated space. Give a maximum of 2/3 of the
+					  space to the completion, and whatever is left to
+					  the description.
+					*/
+					int sum = width[j] - prefix_width - 4 - 2 + (j==cols-1?2:0);
+
+					comp_width = maxi( mini( whole_comp_width,
+											 2*sum/3 ),
+									   sum - whole_desc_width );
+					desc_width = sum-comp_width;
+				}
+
+				/* First we must print the completion. */
+				if( is_quoted )
+				{
+					writestr_ellipsis( el, comp_width);
+				}
+				else
+				{
+					write_escaped_str( el, comp_width );
+				}
+
+				/* Put the description back */
+				*el_end = COMPLETE_SEP;
+
+				/* And print it */
+				set_color( get_color(HIGHLIGHT_PAGER_DESCRIPTION),
+						   FISH_COLOR_IGNORE );
+				writespace( maxi( 2,
+								  width[j]
+								  - comp_width
+								  - desc_width
+								  - 4
+								  - prefix_width
+								  + (j==cols-1?2:0) ) );
+				/* Print description */
+				writestr(L"(");
+				writestr_ellipsis( el_end+1, desc_width);
+				writestr(L")");
+
+				if( j != cols-1)
+					writestr( L"  " );
+
+			}
+		}
+		writech( L'\n' );
+	}
+}
+
+/**
+   Calculates how long the specified string would be when printed on the command line.
+
+   \param str The string to be printed.
+   \param is_quoted Whether the string would be printed quoted or unquoted
+   \param pref_width the preferred width for this item
+   \param min_width the minimum width for this item
+*/
+static void printed_length( wchar_t *str,
+							int is_quoted,
+							int *pref_width,
+							int *min_width )
+{
+	if( is_quoted )
+	{
+		wchar_t *sep = wcschr(str,COMPLETE_SEP);
+		if( sep )
+		{
+			*sep=0;
+			int cw = my_wcswidth( str );
+			int dw = my_wcswidth(sep+1);
+
+			if( termsize.ws_col > 80 )
+				dw = mini( dw, termsize.ws_col/3 );
+
+
+			*pref_width = cw+dw+4;
+
+			if( dw > termsize.ws_col/3 )
+			{
+				dw = termsize.ws_col/3;
+			}
+
+			*min_width=cw+dw+4;
+
+			*sep= COMPLETE_SEP;
+			return;
+		}
+		else
+		{
+			*pref_width=*min_width= my_wcswidth( str );
+			return;
+		}
+
+	}
+	else
+	{
+		int comp_len=0, desc_len=0;
+		int has_description = 0;
+		while( *str != 0 )
+		{
+			switch( *str )
+			{
+				case L'\n':
+				case L'\b':
+				case L'\r':
+				case L'\e':
+				case L'\t':
+				case L'\\':
+				case L'&':
+				case L'$':
+				case L' ':
+				case L'#':
+				case L'^':
+				case L'<':
+				case L'>':
+				case L'@':
+				case L'(':
+				case L')':
+				case L'{':
+				case L'}':
+				case L'?':
+				case L'*':
+				case L'|':
+				case L';':
+				case L':':
+					if( has_description )
+						desc_len++;
+					else
+						comp_len+=2;
+					break;
+
+				case COMPLETE_SEP:
+					has_description = 1;
+					break;
+
+				default:
+					if( has_description )
+						desc_len+= wcwidth(*str);
+					else
+						comp_len+= wcwidth(*str);
+					break;
+			}
+			str++;
+		}
+		if( has_description )
+		{
+			/*
+			  Mangle long descriptions to make formating look nicer
+			*/
+			debug( 3, L"Desc, width = %d %d\n", comp_len, desc_len );
+//			if( termsize.ws_col > 80 )
+//				desc_len = mini( desc_len, termsize.ws_col/3 );
+
+			*pref_width = comp_len+ desc_len+4;;
+
+			comp_len = mini( comp_len, maxi(0,termsize.ws_col/3 - 2));
+			desc_len = mini( desc_len, maxi(0,termsize.ws_col/5 - 4));
+
+			*min_width = comp_len+ desc_len+4;
+			return;
+		}
+		else
+		{
+			debug( 3, L"No desc, width = %d\n", comp_len );
+			
+			*pref_width=*min_width= comp_len;
+			return;
+		}
+
+	}
+}
+
+
+/**
+   Try to print the list of completions l with the prefix prefix using
+   cols as the number of columns. Return 1 if the completion list was
+   printed, 0 if the terminal is to narrow for the specified number of
+   columns. Always succeeds if cols is 1.
+
+   If all the elements do not fit on the screen at once, make the list
+   scrollable using the up, down and space keys to move. The list will
+   exit when any other key is pressed.
+
+   \param cols the number of columns to try to fit onto the screen
+   \param prefix the character string to prefix each completion with
+   \param is_quoted whether the completions should be quoted
+   \param l the list of completions
+
+   \return zero if the specified number of columns do not fit, no-zero otherwise
+*/
+
+static int completion_try_print( int cols,
+								 wchar_t *prefix,
+								 int is_quoted,
+								 array_list_t *l )
+{
+	/*
+	  The calculated preferred width of each column
+	*/
+	int pref_width[32];
+	/*
+	  The calculated minimum width of each column
+	*/
+	int min_width[32];
+	/*
+	  If the list can be printed with this width, width will contain the width of each column
+	*/
+	int *width=pref_width;
+	/*
+	  Set to one if the list should be printed at this width
+	*/
+	int print=0;
+	
+	int i, j;
+	
+	int rows = (al_get_count( l )-1)/cols+1;
+	
+	int pref_tot_width=0;
+	int min_tot_width = 0;
+	int prefix_width = my_wcswidth( prefix );
+	
+	int res=0;
+	/*
+	  Skip completions on tiny terminals
+	*/
+	
+	if( termsize.ws_col < 16 )
+		return 1;
+
+	memset( pref_width, 0, sizeof(pref_width) );
+	memset( min_width, 0, sizeof(min_width) );
+
+	/* Calculated how wide the list would be */
+	for( j = 0; j < cols; j++ )
+	{
+		for( i = 0; i<rows; i++ )
+		{
+			int pref,min;
+			wchar_t *el;
+			if( al_get_count( l ) <= j*rows + i )
+				continue;
+
+			el = (wchar_t *)al_get( l, j*rows + i );
+			printed_length( el, is_quoted, &pref, &min );
+
+			pref += prefix_width;
+			min += prefix_width;
+			if( j != cols-1 )
+			{
+				pref += 2;
+				min += 2;
+			}
+			min_width[j] = maxi( min_width[j],
+								 min );
+			pref_width[j] = maxi( pref_width[j],
+								  pref );
+		}
+		min_tot_width += min_width[j];
+		pref_tot_width += pref_width[j];
+	}
+	/*
+	  Force fit if one column
+	*/
+	if( cols == 1)
+	{
+		if( pref_tot_width > termsize.ws_col )
+		{
+			pref_width[0] = termsize.ws_col;
+		}
+		width = pref_width;
+		print=1;
+	}
+	else if( pref_tot_width <= termsize.ws_col )
+	{
+		/* Terminal is wide enough. Print the list! */
+		width = pref_width;
+		print=1;
+	}
+	else
+	{
+		int next_rows = (al_get_count( l )-1)/(cols-1)+1;
+/*		fwprintf( stderr,
+  L"cols %d, min_tot %d, term %d, rows=%d, nextrows %d, termrows %d, diff %d\n",
+  cols,
+  min_tot_width, termsize.ws_col,
+  rows, next_rows, termsize.ws_row,
+  pref_tot_width-termsize.ws_col );
+*/
+		if( min_tot_width < termsize.ws_col &&
+			( ( (rows < termsize.ws_row) && (next_rows >= termsize.ws_row ) ) ||
+			  ( pref_tot_width-termsize.ws_col< 4 && cols < 3 ) ) )
+		{
+			/*
+			  Terminal almost wide enough, or squeezing makes the whole list fit on-screen
+			*/
+			int tot_width = min_tot_width;
+			width = min_width;
+
+			while( tot_width < termsize.ws_col )
+			{
+				for( i=0; (i<cols) && ( tot_width < termsize.ws_col ); i++ )
+				{
+					if( width[i] < pref_width[i] )
+					{
+						width[i]++;
+						tot_width++;
+					}
+				}
+			}
+			print=1;
+		}
+	}
+
+//	return cols==1;
+	
+	if( print )
+	{
+		res=1;
+		if( rows < termsize.ws_row )
+		{
+			/* List fits on screen. Print it and leave */
+			if( is_ca_mode )
+			{
+				is_ca_mode = 0;
+				writembs(exit_ca_mode);
+			}
+			
+			completion_print( cols, width, 0, rows, prefix, is_quoted, l);
+		}
+		else
+		{
+			int npos, pos = 0;
+			int do_loop = 1;
+
+			is_ca_mode=1;
+			writembs(enter_ca_mode);
+
+			completion_print( cols,
+							  width,
+							  0,
+							  termsize.ws_row-1,
+							  prefix,
+							  is_quoted,
+							  l);
+			/*
+			   List does not fit on screen. Print one screenfull and
+			   leave a scrollable interface
+			*/
+			while(do_loop)
+			{
+				wchar_t msg[10];
+				int percent = 100*pos/(rows-termsize.ws_row+1);
+				set_color( FISH_COLOR_BLACK,
+						   get_color(HIGHLIGHT_PAGER_PROGRESS) );
+				swprintf( msg, 12,
+						  L" %ls(%d%%) \r",
+						  percent==100?L"":(percent >=10?L" ": L"  "),
+						  percent );
+				writestr(msg);
+				set_color( FISH_COLOR_NORMAL, FISH_COLOR_NORMAL );
+				int c = readch();
+
+				switch( c )
+				{
+					case LINE_UP:
+					{
+						if( pos > 0 )
+						{
+							pos--;
+							writembs(tparm( cursor_address, 0, 0));
+							writembs(scroll_reverse);
+							completion_print( cols,
+											  width,
+											  pos,
+											  pos+1,
+											  prefix,
+											  is_quoted,
+											  l );
+							writembs( tparm( cursor_address,
+											 termsize.ws_row-1, 0) );
+							writembs(clr_eol );
+
+						}
+
+						break;
+					}
+
+					case LINE_DOWN:
+					{
+						if( pos <= (rows - termsize.ws_row ) )
+						{
+							pos++;
+							completion_print( cols,
+											  width,
+											  pos+termsize.ws_row-2,
+											  pos+termsize.ws_row-1,
+											  prefix,
+											  is_quoted,
+											  l );
+						}
+						break;
+					}
+
+					case PAGE_DOWN:
+					{
+
+						npos = mini( rows - termsize.ws_row+1,
+									 pos + termsize.ws_row-1 );
+						if( npos != pos )
+						{
+							pos = npos;
+							completion_print( cols,
+											  width,
+											  pos,
+											  pos+termsize.ws_row-1,
+											  prefix,
+											  is_quoted,
+											  l );
+						}
+						else
+						{
+							writembs( flash_screen );
+						}
+
+						break;
+					}
+
+					case PAGE_UP:
+					{
+						npos = maxi( 0,
+									 pos - termsize.ws_row+1 );
+
+						if( npos != pos )
+						{
+							pos = npos;
+							completion_print( cols,
+											  width,
+											  pos,
+											  pos+termsize.ws_row-1,
+											  prefix,
+											  is_quoted,
+											  l );
+						}
+						else
+						{
+							writembs( flash_screen );
+						}
+						break;
+					}
+
+					case R_NULL:
+					{
+						do_loop=0;
+						res=2;
+						break;
+						
+					}
+					
+					default:
+					{
+						sb_append_char( &out_buff, c );
+						do_loop = 0;
+						break;
+					}					
+				}
+			}
+			writembs(clr_eol);
+		}
+	}
+	return res;
+}
+
+/**
+   Substitute any series of tabs, newlines, etc. with a single space character in completion description
+*/
+static void mangle_descriptions( array_list_t *l )
+{
+	int i, skip;
+	for( i=0; i<al_get_count( l ); i++ )
+	{
+		wchar_t *next = (wchar_t *)al_get(l, i);
+		wchar_t *in, *out;
+		skip=0;
+		
+		while( *next != COMPLETE_SEP && *next )
+			next++;
+		
+		if( !*next )
+			continue;
+		
+		in=out=(next+1);
+		
+		while( *in != 0 )
+		{
+			if( *in == L' ' || *in==L'\t' || *in<32 )
+			{
+				if( !skip )
+					*out++=L' ';
+				skip=1;					
+			}
+			else
+			{
+				*out++ = *in;					
+				skip=0;
+			}
+			in++;
+		}
+		*out=0;		
+	}
+}
+
+/**
+   Respond to a winch signal by checking the terminal size
+*/
+static void handle_winch( int sig )
+{
+	if (ioctl(1,TIOCGWINSZ,&termsize)!=0)
+	{
+		return;
+	}
+}
+
+static int interrupt_handler()
+{
+	return R_NULL;
+}
+
+
+static void init()
+{
+	struct sigaction act;
+	program_name = L"fish_pager";
+	fish_setlocale( LC_ALL, L"" );
+	
+	
+	int out = dup( 1 );
+	close(1);
+	if( open( ttyname(0), O_WRONLY ) != 1 )
+	{
+		debug( 0, L"Could not set up file descriptors for pager" );
+		exit( 1 );
+		
+	}
+	out_file = fdopen( out, "w" );
+	sb_init( &out_buff );
+	
+
+			
+	env_universal_init( 0, 0, 0);
+	input_common_init( &interrupt_handler );
+	
+	sigemptyset( & act.sa_mask );
+	act.sa_flags=0;
+	act.sa_handler=SIG_DFL;
+	act.sa_flags = 0;
+	act.sa_handler= &handle_winch;
+	if( sigaction( SIGWINCH, &act, 0 ) )
+	{
+		wperror( L"sigaction" );
+		exit(1);
+	}
+	
+	/* Loop until we are in the foreground.  */
+	while (tcgetpgrp( 0 ) != getpid())
+	{
+		kill (- getpid(), SIGTTIN);
+	}
+
+	/* Put ourselves in our own process group.  */
+	if( getpgrp() != getpid() )
+	{
+		if (setpgid (getpid(), getpid()) < 0)
+		{
+			debug( 1,
+				   L"Couldn't put the shell in its own process group");
+			wperror( L"setpgid" );
+			exit (1);
+		}
+	}
+
+	/* Grab control of the terminal.  */
+	if( tcsetpgrp (STDIN_FILENO, getpid()) )
+	{
+		debug( 1,
+			   L"Couldn't grab control of terminal" );
+		wperror( L"tcsetpgrp" );
+		exit(1);
+	}
+
+	handle_winch( 0 );                /* Set handler for window change events */
+	
+	tcgetattr(0,&pager_modes);        /* get the current terminal modes */
+	memcpy( &saved_modes,
+			&pager_modes,
+			sizeof(saved_modes));     /* save a copy so we can reset the terminal later */
+
+	pager_modes.c_lflag &= ~ICANON;   /* turn off canonical mode */
+	pager_modes.c_lflag &= ~ECHO;     /* turn off echo mode */
+    pager_modes.c_cc[VMIN]=1;
+    pager_modes.c_cc[VTIME]=0;
+
+    if( tcsetattr(0,TCSANOW,&pager_modes))      /* set the new modes */
+    {
+        wperror(L"tcsetattr");
+        exit(1);
+    }
+
+	if( setupterm( 0, STDOUT_FILENO, 0) == ERR )
+	{
+		debug( 0, L"Could not set up terminal" );
+		exit(1);
+	}
+
+	
+//	writembs( tparm( eat_char, ' ', c ) );
+
+}
+
+void destroy()
+{
+	env_universal_destroy();
+	input_common_destroy();
+	del_curterm( cur_term );
+	sb_destroy( &out_buff );
+	fclose( out_file );
+
+}
+
+int main( int argc, char **argv )
+{
+	int i;
+	int is_quoted=0;	
+	array_list_t comp;
+	wchar_t *prefix;
+
+	
+	init();
+
+	
+	prefix = str2wcs( argv[2] );
+	is_quoted = strcmp( "1", argv[1] )==0;
+	
+	debug( 3, L"prefix is '%ls'", prefix );
+	
+	al_init( &comp );
+	
+	for( i=3; i<argc; i++ )
+	{
+		al_push( &comp, str2wcs( argv[i] ) );
+	}
+	
+	mangle_descriptions( &comp );
+
+	for( i = 6; i>0; i-- )
+	{
+		switch( completion_try_print( i, prefix, is_quoted, &comp ) )
+		{
+			case 0:
+				break;
+			case 1:
+				i=0;
+				break;
+			case 2:
+				i=7;
+				break;
+		}
+		
+	}
+	
+	al_foreach( &comp, (void(*)(const void *))&free );
+	al_destroy( &comp );	
+	free(prefix );
+
+	fwprintf( out_file, L"%ls", (wchar_t *)out_buff.buff );
+	if( is_ca_mode )
+		writembs(exit_ca_mode);
+
+	
+	destroy();
+}
+

fish_tests.c

@@ -0,0 +1,647 @@
+/** \file main.c
+	Various bug and feature tests. Compiled and run by make test.
+*/
+
+#include "config.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <wchar.h>
+#include <string.h>
+#include <unistd.h>
+#include <errno.h>
+#include <unistd.h>
+#include <termios.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <stdarg.h>		
+
+#ifdef HAVE_GETOPT_H
+#include <getopt.h>
+#endif
+
+#include <signal.h>
+
+#include <locale.h>
+#include <dirent.h>
+
+#include "util.h"
+#include "common.h"
+#include "proc.h"
+#include "reader.h"
+#include "builtin.h"
+#include "function.h"
+#include "complete.h"
+#include "wutil.h"
+#include "env.h"
+#include "expand.h"
+#include "parser.h"
+#include "tokenizer.h"
+
+#define LAPS 50
+
+static int err_count=0;
+
+static void say( wchar_t *blah, ... )
+{
+	va_list va;
+	va_start( va, blah );
+	vwprintf( blah, va );
+	va_end( va );	
+	wprintf( L"\n" );
+}
+
+static void err( wchar_t *blah, ... )
+{
+	va_list va;
+	va_start( va, blah );
+	err_count++;
+	
+	wprintf( L"Error: " );
+	vwprintf( blah, va );
+	va_end( va );	
+	wprintf( L"\n" );
+}
+
+static void ok()
+{
+	wprintf( L"OK\n" );
+}
+
+
+static int pq_compare( void *e1, void *e2 )
+{
+	return e1-e2;
+}
+
+static int pq_test( int elements )
+{
+	int i;
+	int prev;
+	
+	int *count = calloc( sizeof(int), 100 );
+	
+	priority_queue_t q;
+	pq_init( &q, pq_compare );
+
+	
+	for( i=0; i<elements; i++ )
+	{
+		int foo = rand() % 100;
+//		printf( "Adding %d\n", foo );
+		pq_put( &q, (void *)foo );
+		count[foo]++;
+	}
+	
+	prev = 100;
+	
+	for( i=0; i<elements; i++ )
+	{
+		int pos = (int)pq_get( &q );
+		count[ pos ]--;
+		if( pos > prev )
+			err( L"Wrong order of elements in priority_queue_t" );
+		prev = pos;
+		
+	}
+
+	for( i=0; i<100; i++ )
+	{
+		if( count[i] != 0 )
+		{
+			err( L"Wrong number of elements in priority_queue_t" );
+		}
+	}
+}
+
+
+static int stack_test( int elements )
+{
+	int i;
+
+	int res=1;
+	
+	array_list_t s;
+
+	al_init( &s );	
+
+	for( i=0; i<elements; i++ )
+	{
+		int foo;
+
+		al_push( &s, (void*)i);
+		al_push( &s, (void*)i);
+		
+		if( (foo=(int)al_pop( &s )) != i )
+		{
+			err( L"Unexpected data" );
+			res = 0;			
+			break;
+		}
+	}
+
+	for( i=0; i<elements; i++ )
+	{
+		int foo;
+		
+		if( (foo=(int)al_pop( &s )) != (elements-i-1) )
+		{
+			err( L"Unexpected data" );
+			res = 0;
+			break;
+		}
+		
+		
+	}
+
+
+	al_destroy( &s );
+	
+	return res;
+}
+
+
+static int hash_func( const void *data )
+{
+/*	srand( (int)data );
+	return rand();
+*/
+	int foo = (int)data;
+	return 127*((foo^0xefc7e214)) ^(foo<<11);	
+}
+
+static int compare_func( const void *key1, const void *key2 )
+{
+	return key1==key2;
+}
+
+
+static int hash_test( int elements )
+{
+	int i;
+	int res=1;
+		
+	hash_table_t h;
+
+	hash_init( &h, hash_func, compare_func );
+	
+	for( i=1; i< elements+1; i++ )
+	{
+		hash_put( &h, (void*)i, (void*)100-i );
+	}
+	
+	for( i=1; i< elements+1; i++ )
+	{
+		if( (int)hash_get( &h, (void*)i ) != (100-i) )
+		{
+			err( L"Key %d gave data %d, expected data %d",
+				 i, 
+				 (int)hash_get( &h, (void*)i ),
+				 100-i );
+			res = 0;
+			
+			break;
+		}
+	}
+
+	if( hash_get_count( &h ) != elements )
+	{
+		err( L"Table holds %d elements, should hold %d elements",
+			 hash_get_count( &h ),
+			 elements );
+		res = 0;
+		
+	}
+	
+	
+	for( i=1; i<elements+1; i+=2 )
+	{
+		hash_remove( &h, (void*)i, 0, 0 );
+				
+	}
+
+	if( hash_get_count( &h ) != ((elements)/2) )
+	{
+		err( L"Table contains %d elements, should contain %d elements",
+			 hash_get_count( &h ),
+			 elements/2 );
+		res = 0;
+	}
+	
+	for( i=1; i<elements+1; i++ )
+	{
+		if( hash_contains( &h, (void*)i) != (i+1)%2 )
+		{
+			if( i%2 )
+				err( L"Key %d remains, should be deleted",
+						i );
+			else
+				err( L"Key %d does not exist",
+					 i );
+			res = 0;
+			break;
+			}
+	}
+
+	hash_destroy( &h );
+
+	return res;
+	
+}
+
+static int al_test( int sz)
+{
+	int i;	
+	array_list_t l;
+
+	
+
+	al_init( &l );
+	
+	al_set( &l, 1, (void *)7 );
+	al_set( &l, sz, (void *)7 );
+	
+	if( al_get_count( &l ) != maxf( sz+1, 2 ) )
+		err( L"Wrong number of elements in array list" );
+	
+	for( i=0; i<al_get_count( &l ); i++ )
+	{
+		int val = (int)((long) al_get( &l, i ));
+		if( (i == 1) || (i==sz))
+		{
+			if( val != 7 )
+				err( L"Canary changed to %d at index %d", val, i );
+		}
+		else
+		{
+			if( val != 0 )
+				err( L"False canary %d found at index %d", val, i );
+		}
+	}
+}
+
+static void sb_test()
+{
+	string_buffer_t b;
+	int res;
+	
+	sb_init( &b );
+	
+	if( res=sb_printf( &b, L"%ls%s", L"Testing ", "string_buffer_t " ) == -1 )
+	{
+		err( L"Error %d while testing stringbuffers", res );
+	}
+	
+	if( (res=sb_printf( &b, L"%ls", L"functionality" ))==-1)	
+	{
+		err( L"Error %d while testing stringbuffers", res );
+	}
+	say( (wchar_t *)b.buff );
+}
+
+
+static void test_util()
+{
+	int i;
+
+	say( L"Testing utility library" );
+	
+	for( i=0; i<18; i++ )
+	{
+		long t1, t2;
+		pq_test( 1<<i );
+		stack_test( 1<<i );
+		t1 = get_time();
+		hash_test( 1<<i );
+		t2 = get_time();
+		if( i > 8 )
+			say( L"Hashtable uses %f microseconds per element at size %d",
+				 ((double)(t2-t1))/(1<<i),
+				1<<i );
+		al_test( 1<<i );
+	}
+
+	sb_test();
+	
+	
+/*
+	int i;
+	for( i=2; i<10000000; i*=2 )
+	{
+		
+		printf( "%d", i );
+		
+		t1 = get_time();
+		
+		if(!hash_test(i))
+			exit(0);
+	
+		t2 = get_time();
+		
+		printf( " %d\n", (t2-t1)/i );
+		
+		
+	}
+*/	
+}
+
+
+
+
+static void test_tok()
+{
+	tokenizer t;
+	
+	say( L"Testing tokenizer" );
+
+	
+	say( L"Testing invalid input" );
+	tok_init( &t, 0, 0 );
+
+	if( tok_last_type( &t ) != TOK_ERROR )
+	{
+		err(L"Invalid input to tokenizer was undetected" );
+	}
+	
+	say( L"Testing use of broken tokenizer" );
+	if( !tok_has_next( &t ) )
+	{
+		err( L"tok_has_next() should return 1 once on broken tokenizer" );
+	}
+	
+	tok_next( &t );
+	if( tok_last_type( &t ) != TOK_ERROR )
+	{
+		err(L"Invalid input to tokenizer was undetected" );
+	}
+
+	/*
+	  This should crash if there is a bug. No reliable way to detect otherwise.
+	*/
+	say( L"Test destruction of broken tokenizer" );
+	tok_destroy( &t );
+	
+	{
+
+		wchar_t *str = L"string <redirection  2>&1 'nested \"quoted\" '(string containing subshells ){and,brackets}$as[$well (as variable arrays)]";
+		const int types[] = 
+		{
+			TOK_STRING, TOK_REDIRECT_IN, TOK_STRING, TOK_REDIRECT_FD, TOK_STRING, TOK_STRING, TOK_END
+		}
+		;
+		int i;
+		
+		say( L"Test correct tokenization" );
+
+		for( i=0, tok_init( &t, str, 0 ); i<(sizeof(types)/sizeof(int)); i++,tok_next( &t ) )
+		{
+			if( types[i] != tok_last_type( &t ) )
+			{
+				err( L"Tokenization error:");
+				wprintf( L"Token number %d of string \n'%ls'\n, expected token type %ls, got token '%ls' of type %ls\n", 
+						 i+1,
+						 str,
+						 tok_get_desc(types[i]),
+						 tok_last(&t),
+						 tok_get_desc(tok_last_type( &t )) );
+			}
+		}
+	}
+	
+
+		
+}
+
+static void test_parser()
+{
+	say( L"Testing parser" );
+	
+	
+	say( L"Testing null input to parser" );
+	if( !parser_test( 0, 0 ) )
+	{
+		err( L"Null input to parser_test undetected" );
+	}
+
+	say( L"Testing block nesting" );
+	if( !parser_test( L"if; end", 0 ) )
+	{
+		err( L"Incomplete if statement undetected" );			
+	}
+	if( !parser_test( L"if test; echo", 0 ) )
+	{
+		err( L"Missing end undetected" );			
+	}
+	if( !parser_test( L"if test; end; end", 0 ) )
+	{
+		err( L"Unbalanced end undetected" );			
+	}
+
+	say( L"Testing detection of invalid use of builtin commands" );
+	if( !parser_test( L"case foo", 0 ) )
+	{
+		err( L"'case' command outside of block context undetected" );		
+	}
+	if( !parser_test( L"switch ggg; if true; case foo;end;end", 0 ) )
+	{
+		err( L"'case' command outside of switch block context undetected" );		
+	}
+	if( !parser_test( L"else", 0 ) )
+	{
+		err( L"'else' command outside of conditional block context undetected" );		
+	}
+	if( !parser_test( L"break", 0 ) )
+	{
+		err( L"'break' command outside of loop block context undetected" );		
+	}
+	if( !parser_test( L"exec ls|less", 0 ) || !parser_test( L"echo|return", 0 ))
+	{
+		err( L"Invalid pipe command undetected" );		
+	}	
+
+	say( L"Testing basic evaluation" );
+	if( !eval( 0, 0, TOP ) )
+	{
+		err( L"Null input when evaluating undetected" );
+	}	
+	if( !eval( L"ls", 0, WHILE ) )
+	{
+		err( L"Invalid block mode when evaluating undetected" );
+	}
+	
+}
+	
+/**
+   Perform parameter expantion and test if the output equals the zero-terminated parameter list supplied.
+
+   \param in the string to expand
+   \param flags the flags to send to expand_string
+*/
+
+static int expand_test( const wchar_t *in, int flags, ... )
+{
+	array_list_t out;	
+	va_list va;
+	int i=0;
+	int res=1;
+	wchar_t *arg;
+	
+	al_init( &out );
+	expand_string( wcsdup(in), &out, flags);
+	
+	va_start( va, flags );
+
+	while( (arg=va_arg(va, wchar_t *) )!= 0 ) 
+	{
+		if( al_get_count( &out ) == i )
+		{
+			res=0;
+			break;
+		}
+		
+		if( wcscmp( al_get( &out, i ),arg) != 0 )
+		{
+			res=0;
+			break;
+		}
+		
+		i++;		
+	}
+	va_end( va );
+	
+	al_foreach( &out, (void (*)(const void *))&free );
+	return res;
+		
+}
+
+
+static void test_expand()
+{
+	say( L"Testing parameter expantion" );
+	
+	if( !expand_test( L"foo", 0, L"foo", 0 ))
+	{
+		err( L"Strings do not expand to themselves" );
+	}
+
+	if( !expand_test( L"a{b,c,d}e", 0, L"abe", L"ace", L"ade", 0 ) )
+	{
+		err( L"Bracket expantion is broken" );
+	}
+
+	if( !expand_test( L"a*", EXPAND_SKIP_WILDCARDS, L"a*", 0 ) )
+	{
+		err( L"Cannot skip wildcard expantion" );
+	}
+	
+}
+
+
+void perf_complete()
+{
+	wchar_t c;
+	array_list_t out;
+	long long t1, t2;
+	int matches=0;
+	double t;
+	wchar_t str[3]=
+	{
+		0, 0, 0 
+	}
+	;
+	int i;
+	
+	
+	say( L"Testing completion performance" );
+	al_init( &out );
+	
+	reader_push(L"");
+	say( L"Here we go" );
+	 	
+	t1 = get_time();
+	
+	
+	for( c=L'a'; c<=L'z'; c++ )
+	{
+		str[0]=c;
+		reader_set_buffer( str, 0 );
+		
+		complete( str, &out );
+	
+		matches += al_get_count( &out );
+		
+		al_foreach( &out, (void (*)(const void *))&free );
+		al_truncate( &out, 0 );
+	}
+	t2=get_time();
+	
+	t = (double)(t2-t1)/(1000000*26);
+	
+	say( L"One letter command completion took %f seconds per completion, %f microseconds/match", t, (double)(t2-t1)/matches );
+	
+	matches=0;
+	t1 = get_time();
+	for( i=0; i<LAPS; i++ )
+	{
+		str[0]='a'+(rand()%26);
+		str[1]='a'+(rand()%26);
+		
+		reader_set_buffer( str, 0 );
+		
+		complete( str, &out );
+	
+		matches += al_get_count( &out );
+		
+		al_foreach( &out, (void (*)(const void *))&free );
+		al_truncate( &out, 0 );
+	}
+	t2=get_time();
+	
+	t = (double)(t2-t1)/(1000000*LAPS);
+	
+	say( L"Two letter command completion took %f seconds per completion, %f microseconds/match", t, (double)(t2-t1)/matches );
+	
+	al_destroy( &out );
+
+	reader_pop();
+	
+}
+
+
+int main( int argc, char **argv )
+{
+
+	say( L"Testing low-level functionality");
+	say( L"Lines beginning with 'fish:' are not errors, they are warning messages\ngenerated by the fish parser library when given broken input, and can be\nignored. All errors begin with 'Error:'." );
+	
+	parser_init();
+	function_init();
+	builtin_init();
+	env_init();
+	complete_init();
+	reader_init();
+	
+	test_util();
+	test_tok();
+	test_parser();
+	test_expand();
+		
+	say( L"Encountered %d errors in low-level tests", err_count );
+
+	/*
+	  Skip performance tests for now, since they seem to hang when running from inside make (?)
+	*/
+//	say( L"Testing performance" );
+//	perf_complete();
+		
+	reader_destroy();
+	
+	parser_destroy();
+	function_destroy();
+	builtin_destroy();
+	env_destroy();
+	complete_destroy();
+	wutil_destroy();
+	
+}

fishd.c

@@ -0,0 +1,443 @@
+/** \file fishd.c
+
+The universal variable server. fishd is automatically started by fish
+if a fishd server isn't already running. fishd reads any saved
+variables from ~/.fishd, and takes care of commonication between fish
+instances. When no clients are running, fishd will automatically shut
+down and save.
+
+*/
+#include <stdio.h>
+#include <stdlib.h>
+#include <wchar.h>
+#include <unistd.h>
+#include <errno.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/un.h>
+#include <pwd.h>
+#include <fcntl.h>
+
+#include <errno.h>
+#include <locale.h>
+#include <signal.h>
+
+#include "util.h"
+#include "common.h"
+#include "wutil.h"
+#include "env_universal_common.h"
+
+
+#ifndef UNIX_PATH_MAX 
+#define UNIX_PATH_MAX 100
+#endif
+
+#define GREETING "#Fish universal variable daemon\n#Lines beginning with '#' are ignored\n#Syntax:\n#SET VARNAME:VALUE\n#or\n#ERASE VARNAME\n#Where VALUE is the escaped value of the variable\n#Backslash escapes and \\xxx hexadecimal style escapes are supported\n"
+#define FILE ".fishd"
+
+
+static connection_t *conn;
+static int sock;
+
+
+int get_socket()
+{
+	int s, len;
+	struct sockaddr_un local;
+	char *name;
+	char *dir = getenv( "FISHD_SOCKET_DIR" );
+	char *uname = getenv( "USER" );
+
+	if( !dir )
+		dir = "/tmp";
+	if( uname==0 )
+	{
+		struct passwd *pw;
+		pw = getpwuid( getuid() );
+		uname = strdup( pw->pw_name );
+	}
+		
+	name = malloc( strlen(dir)+ strlen(uname)+ strlen(SOCK_FILENAME) + 2 );
+	strcpy( name, dir );
+	strcat( name, "/" );
+	strcat( name, SOCK_FILENAME );
+	strcat( name, uname );
+
+	if( strlen( name ) >= UNIX_PATH_MAX )
+	{
+		debug( 1, L"Filename too long: '%s'", name );
+		exit(1);
+	}
+	
+	debug( 1, L"Connect to socket at %s", name );
+	
+	if ((s = socket(AF_UNIX, SOCK_STREAM, 0)) == -1) {
+		perror("socket");
+		exit(1);
+	}
+	
+	local.sun_family = AF_UNIX;
+	strcpy(local.sun_path, name );
+	unlink(local.sun_path);
+	len = strlen(local.sun_path) + sizeof(local.sun_family);
+
+	free( name );
+	
+	if (bind(s, (struct sockaddr *)&local, len) == -1) {
+		perror("bind");
+		exit(1);
+	}
+/*		
+	if( setsockopt( s, SOL_SOCKET, SO_PASSCRED, &on, sizeof( on ) ) )
+	{
+		perror( "setsockopt");
+		exit(1);
+	}
+*/
+
+	if( fcntl( s, F_SETFL, O_NONBLOCK ) != 0 )
+	{
+		wperror( L"fcntl" );
+		close( s );		
+		return -1;
+
+	}
+	
+	return s;
+}
+
+void enqueue( const void *k,
+			  const void *v,
+			  void *q)
+{
+	const wchar_t *key = (const wchar_t *)k;
+	const wchar_t *val = (const wchar_t *)v;
+	queue_t *queue = (queue_t *)q;
+	
+	message_t *msg = create_message( SET, key, val );
+	msg->count=1;
+	
+	q_put( queue, msg );
+}
+
+void enqueue_all( connection_t *c )
+{
+	hash_foreach2( &env_universal_var,
+	               &enqueue, 
+	               (void *)&c->unsent );
+	try_send_all( c );
+}
+
+void broadcast( int type, const wchar_t *key, const wchar_t *val )
+{
+	connection_t *c;
+	message_t *msg;
+
+	if( !conn )
+		return;
+	
+	msg = create_message( type, key, val );
+
+	/*
+	  Don't merge loops, or try_send_all can free the message prematurely
+	*/
+			
+	for( c = conn; c; c=c->next )
+	{
+		msg->count++;
+		q_put( &c->unsent, msg );
+	}	
+
+	for( c = conn; c; c=c->next )
+	{
+		try_send_all( c );
+	}	
+}
+
+void daemonize()
+{
+	/*
+	  Fork, and let parent exit
+	*/
+	switch( fork() )
+	{
+		case -1:
+			debug( 0, L"Could not put fishd in background. Quitting" );
+			wperror( L"fork" );
+			exit(1);
+			
+		case 0:
+		{
+			struct sigaction act;
+			sigemptyset( & act.sa_mask );
+			act.sa_flags=0;
+			act.sa_handler=SIG_IGN;
+			sigaction( SIGHUP, &act, 0);
+			break;
+		}
+		
+		default:
+		{			
+			debug( 0, L"Parent calling exit" );
+			exit(0);
+		}		
+	}
+
+	/*
+	  Put ourself in out own processing group
+	*/
+	setpgrp();
+
+	/*
+	  Close stdin and stdout
+	*/
+	close( 0 );
+	close( 1 );
+
+}
+
+
+void load()
+{
+	struct passwd *pw;
+	char *name;
+	char *dir = getenv( "HOME" );
+	if( !dir )
+	{
+		pw = getpwuid( getuid() );
+		dir = pw->pw_dir;
+	}
+		
+	name = malloc( strlen(dir)+ strlen(FILE)+ 2 );
+	strcpy( name, dir );
+	strcat( name, "/" );
+	strcat( name, FILE );
+	
+	debug( 1, L"Open file for loading: '%s'", name );
+	
+	connection_t load;
+	load.fd = open( name, O_RDONLY);
+	
+	free( name );
+	
+	if( load.fd == -1 )
+	{
+		debug( 0, L"Could not open save file. No previous saves?" );
+	}
+	debug( 1, L"Load input file on fd %d", load.fd );
+	sb_init( &load.input );
+	memset (&load.wstate, '\0', sizeof (mbstate_t));
+	read_message( &load );
+	sb_destroy( &load.input );
+	close( load.fd );
+}
+
+void save()
+{
+	struct passwd *pw;
+	char *name;
+	char *dir = getenv( "HOME" );
+	if( !dir )
+	{
+		pw = getpwuid( getuid() );
+		dir = pw->pw_dir;
+	}
+		
+	name = malloc( strlen(dir)+ strlen(FILE)+ 2 );
+	strcpy( name, dir );
+	strcat( name, "/" );
+	strcat( name, FILE );
+	
+	debug( 1, L"Open file for saving: '%s'", name );
+	
+	connection_t save;
+	save.fd = open( name, O_CREAT | O_TRUNC | O_WRONLY);
+	free( name );
+
+	if( save.fd == -1 )
+	{
+		debug( 0, L"Could not open save file" );
+		wperror( L"open" );
+		exit(1);
+	}
+	debug( 1, L"File open on fd %d'", save.fd );
+	q_init( &save.unsent );
+	enqueue_all( &save );
+	close( save.fd );
+	q_destroy( &save.unsent );
+}
+
+static void init()
+{
+	program_name=L"fishd";
+	sock = get_socket();
+	if (listen(sock, 64) == -1) 
+	{
+		wperror(L"listen");
+		exit(1);
+	}	
+
+	daemonize();	
+
+	fish_setlocale( LC_ALL, L"" );
+	
+	env_universal_common_init( &broadcast );
+	
+	load();	
+}
+
+int main( int argc, char ** argv )
+{
+	int child_socket, t;
+	struct sockaddr_un remote;
+	int max_fd;
+	int update_count=0;
+	
+	fd_set read_fd, write_fd;
+	
+	init();
+	
+	while(1) 
+	{
+		connection_t *c;
+		int res;
+
+		t=sizeof( remote );		
+		
+		FD_ZERO( &read_fd );
+		FD_ZERO( &write_fd );
+		FD_SET( sock, &read_fd );
+		max_fd = sock+1;
+		for( c=conn; c; c=c->next )
+		{
+			FD_SET( c->fd, &read_fd );
+			max_fd = maxi( max_fd, c->fd+1);
+			
+			if( ! q_empty( &c->unsent ) )
+			{
+				FD_SET( c->fd, &write_fd );
+			}
+		}
+		
+		res=select( max_fd, &read_fd, &write_fd, 0, 0 );
+		
+		if( res==-1 )
+		{
+			wperror( L"select" );
+			exit(1);
+		}
+		
+		if( FD_ISSET( sock, &read_fd ) )
+		{
+			if( (child_socket = 
+				 accept( sock, 
+						 (struct sockaddr *)&remote, 
+						 &t) ) == -1) {
+				wperror( L"accept" );
+				exit(1);
+			}
+			else
+			{
+				debug( 1, L"Connected with new child on fd %d", child_socket );
+
+				if( fcntl( child_socket, F_SETFL, O_NONBLOCK ) != 0 )
+				{
+					wperror( L"fcntl" );
+					close( child_socket );		
+				}
+				else
+				{
+					connection_t *new = malloc( sizeof(connection_t));
+					new->fd = child_socket;
+					new->next = conn;
+					q_init( &new->unsent );
+					new->killme=0;				
+					sb_init( &new->input );
+					memset (&new->wstate, '\0', sizeof (mbstate_t));					
+					send( new->fd, GREETING, strlen(GREETING), MSG_DONTWAIT );
+					enqueue_all( new );				
+					conn=new;
+				}
+			}
+		}
+		
+		for( c=conn; c; c=c->next )
+		{
+			if( FD_ISSET( c->fd, &write_fd ) )
+			{
+				try_send_all( c );
+			}
+		}
+		
+		for( c=conn; c; c=c->next )
+		{
+			if( FD_ISSET( c->fd, &read_fd ) )
+			{
+				read_message( c );
+
+				/*
+				  Occasionally we save during normal use, so that we
+				  won't lose everything on a system crash
+				*/
+				update_count++;
+				if( update_count >= 8 )
+				{
+					save();
+					update_count = 0;
+				}
+			}
+		}
+		
+		connection_t *prev=0;
+		c=conn;
+		
+		while( c )
+		{
+			if( c->killme )
+			{
+				debug( 1, L"Close connection %d", c->fd );
+
+				close(c->fd );
+				sb_destroy( &c->input );
+
+				while( !q_empty( &c->unsent ) )
+				{
+					message_t *msg = (message_t *)q_get( &c->unsent );
+					msg->count--;
+					if( !msg->count )
+						free( msg );
+				}
+				
+				q_destroy( &c->unsent );				
+				if( prev )
+				{
+					prev->next=c->next;
+				}
+				else
+				{
+					conn=c->next;
+				}
+				
+				free(c);
+				
+				c=(prev?prev->next:conn);
+				
+			}
+			else
+			{
+				prev=c;
+				c=c->next;
+			}
+		}
+		if( !conn )
+		{
+			debug( 0, L"No more clients. Quitting" );
+			save();			
+			env_universal_common_destroy();
+			exit(0);
+			c=c->next;
+		}
+		
+	}
+}

function.c

@@ -0,0 +1,144 @@
+/** \file function.c
+  Functions for storing and retrieving function information.
+*/
+#include <stdlib.h>
+#include <stdio.h>
+#include <wchar.h>
+#include <unistd.h>
+#include <termios.h>
+#include <signal.h>
+
+
+#include "config.h"
+#include "util.h"
+#include "function.h"
+#include "proc.h"
+#include "parser.h"
+#include "common.h"
+#include "intern.h"
+
+/**
+   Table containing all functions
+*/
+static hash_table_t function;
+
+/**
+   Struct describing a function
+*/
+typedef struct
+{
+	/** Function definition */
+	wchar_t *cmd;
+	/** Function description */
+	wchar_t *desc;	
+}
+function_data_t;
+
+/**
+   Free all contents of an entry to the function hash table
+*/
+static void clear_function_entry( const void *key, 
+							   const void *data )
+{
+	function_data_t *d = (function_data_t *)data;
+	free( (void *)d->cmd );
+	free( (void *)d->desc );
+	free( (void *)d );
+}
+
+void function_init()
+{
+	hash_init( &function,
+			   &hash_wcs_func,
+			   &hash_wcs_cmp );
+}
+
+void function_destroy()
+{
+	hash_foreach( &function, &clear_function_entry );
+	hash_destroy( &function );
+}
+
+void function_add( const wchar_t *name, 
+				   const wchar_t *val,
+				   const wchar_t *desc )
+{
+	if( function_exists( name ) )
+		function_remove( name );
+	
+	function_data_t *d = malloc( sizeof( function_data_t ) );
+	d->cmd = wcsdup( val );
+	d->desc = desc?wcsdup( desc ):0;
+	hash_put( &function, intern(name), d );
+}
+
+int function_exists( const wchar_t *cmd )
+{
+	return (hash_get(&function, cmd) != 0 );
+}
+
+void function_remove( const wchar_t *name )
+{
+	void *key;
+	function_data_t *d;
+
+	hash_remove( &function,
+				 name,
+				 (const void **) &key,
+				 (const void **)&d );
+
+	if( !d )
+		return;
+
+	clear_function_entry( key, d );
+}
+	
+const wchar_t *function_get_definition( const wchar_t *argv )
+{
+	function_data_t *data = 
+		(function_data_t *)hash_get( &function, argv );
+	if( data == 0 )
+		return 0;
+	return data->cmd;
+}
+	
+const wchar_t *function_get_desc( const wchar_t *argv )
+{
+	function_data_t *data = 
+		(function_data_t *)hash_get( &function, argv );
+	if( data == 0 )
+		return 0;
+	
+	return data->desc?data->desc:data->cmd;
+}
+
+void function_set_desc( const wchar_t *name, const wchar_t *desc )
+{
+	function_data_t *data = 
+		(function_data_t *)hash_get( &function, name );
+	if( data == 0 )
+		return;
+
+	data->desc =wcsdup(desc);
+}
+
+/**
+   Helper function for removing hidden functions 
+*/
+static void get_names_internal( const void *key,
+								const void *val,
+								void *aux )
+{
+	wchar_t *name = (wchar_t *)key;
+	if( name[0] != L'_' )
+		al_push( (array_list_t *)aux, name );
+}
+
+void function_get_names( array_list_t *list, int get_hidden )
+{
+	if( get_hidden )
+		hash_get_keys( &function, list );
+	else
+		hash_foreach2( &function, &get_names_internal, list );
+	
+}

function.h

@@ -0,0 +1,63 @@
+/** \file function.h 
+
+    Prototypes for functions for storing and retrieving function
+	information. Actual function evaluation is taken care of by the
+	parser and to some degree the builtin handling library.
+*/
+
+/**
+   Initialize function data   
+*/
+void function_init();
+/**
+   Destroy function data
+*/
+void function_destroy();
+
+/**
+   Add an function. The parameters values are copied and should be freed by the caller.
+*/
+void function_add( const wchar_t *name,
+				   const wchar_t *val,
+				   const wchar_t *desc );
+
+/**
+   Remove the function with the specified name.
+*/
+void function_remove( const wchar_t *name );
+
+/**
+   Returns true if the function with the name name uses internal variables, false otherwise.
+*/
+int function_use_vars( const wchar_t *name );
+
+/**
+   Returns the definition of the function with the name \c name.
+*/
+const wchar_t *function_get_definition( const wchar_t *name );
+
+/**
+   Returns the description of the function with the name \c name.
+*/
+const wchar_t *function_get_desc( const wchar_t *name );
+
+/**
+   Sets the description of the function with the name \c name.
+*/
+void function_set_desc( const wchar_t *name, const wchar_t *desc );
+
+/**
+   Returns true if the function witrh the name name exists.
+*/
+int function_exists( const wchar_t *name);
+
+/**
+   Insert all function names into l. These are not copies of the strings and should not be freed after use.
+   
+   \param list the list to add the names to
+   \param get_hidden whether to include hidden functions, i.e. ones starting with an underscore
+*/
+void function_get_names( array_list_t *list, 
+						 int get_hidden );
+
+

gen_hdr.sh

@@ -0,0 +1,12 @@
+#!/bin/sh
+
+# This little script calls the man command to render a manual page and
+# pipes the result into the gen_hdr2 program to convert the output
+# into a C string literal. 
+
+# NAME is the name of the function we are generating documentation for.
+NAME=$(basename $1 .doxygen)
+
+# Render the page
+nroff -man doc_src/builtin_doc/man/man1/${NAME}.1 | col -b | cat -s | sed -e '$d' | ./gen_hdr2
+

gen_hdr2.c

@@ -0,0 +1,76 @@
+/** \file gen_hdr2.c
+  A program that reads data from stdin and outputs it as a C string.
+
+   It is used as a part of the build process to generate help texts
+   for the built in commands.
+*/
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+/**
+   The main function, does all the work
+*/
+int main()
+{
+	int line = 0;
+	printf( "\t\t\"" );
+	int c;
+	int count=0;
+	while( (c=getchar()) != EOF )
+	{
+		if( c == '\n' )
+			line++;
+		
+		if( line > 4 )
+			break;
+	} 
+	
+	while( (c=getchar()) != EOF )
+	{
+		if( (c >= 'a' && c <= 'z' ) || 
+			(c >= 'A' && c <= 'Z' ) || 
+			(c >= '0' && c <= '9' ) ||
+			( strchr(" ,.!;:-_#$%&(){}[]<>=?+-*/'",c) != 0) )
+		{
+			count++;
+			putchar(c);
+		}
+		else
+		{
+			switch(c)
+			{
+				case '\n':
+					printf( "\\n" );
+					printf( "\"\n\t\t\"" );
+					count =0;
+					break;
+				case '\t':
+					printf( "\\t" );
+					count +=2;
+					break;
+				case '\r':
+					printf( "\\r" );
+					count +=2;
+					break;
+					
+				case '\"':
+				case '\\':
+					printf( "\\%c",c );
+					count +=2;
+					break;
+					
+				default:
+					count +=7;
+					printf( "\\x%02x\" \"", c );
+					break;
+			}
+		}
+		if( count > 60 )
+		{
+			count=0;
+			printf( "\"\n\t\t\"" );
+		}
+	}
+	printf( "\"" );
+	return 0;	
+}

highlight.c

@@ -0,0 +1,557 @@
+/** \file highlight.c
+	Functions for syntax highlighting
+*/
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <dirent.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <errno.h>
+#include <wchar.h>
+#include <wctype.h>
+#include <sys/types.h>
+#include <termios.h>
+#include <signal.h>
+
+#include "config.h"
+#include "util.h"
+#include "wutil.h"
+#include "highlight.h"
+#include "tokenizer.h"
+#include "proc.h"
+#include "parser.h"
+#include "builtin.h"
+#include "function.h"
+#include "env.h"
+#include "expand.h"
+#include "sanity.h"
+#include "common.h"
+#include "complete.h"
+#include "output.h"
+
+static void highlight_universal_internal( wchar_t * buff, 
+										  int *color, 
+										  int pos, 
+										  array_list_t *error );
+
+
+/**
+   The environment variables used to specify the color of different tokens.
+*/
+static wchar_t *hightlight_var[] = 
+{
+	L"fish_color_normal",
+	L"fish_color_command",
+	L"fish_color_subshell",
+	L"fish_color_redirection",
+	L"fish_color_end",
+	L"fish_color_error",
+	L"fish_color_param",
+	L"fish_color_comment",
+	L"fish_color_match",
+	L"fish_color_search_match",
+	L"fish_color_pager_prefix",
+	L"fish_color_pager_completion",
+	L"fish_color_pager_description",
+	L"fish_color_pager_progress"
+}
+	;
+
+
+int highlight_get_color( int highlight )
+{
+	if( highlight < 0 )
+		return FISH_COLOR_NORMAL;
+	if( highlight >= (12) )
+		return FISH_COLOR_NORMAL;
+	
+	wchar_t *val = env_get( hightlight_var[highlight]);
+	if( val == 0 )
+		val = env_get( hightlight_var[HIGHLIGHT_NORMAL]);
+	
+	if( val == 0 )
+	{
+		return FISH_COLOR_NORMAL;
+	}
+	
+	int i;
+	int color;
+
+	return output_color_code( val );
+	
+}
+
+
+void highlight_shell( wchar_t * buff, 
+					  int *color, 
+					  int pos, 
+					  array_list_t *error )
+{
+	tokenizer tok;
+	int had_cmd=0;
+	int i;
+	int last_val;
+	wchar_t *last_cmd=0;
+	int len = wcslen(buff);
+	
+	if( !len )
+		return;
+	
+	for( i=0; buff[i] != 0; i++ )
+		color[i] = -1;
+	
+	for( tok_init( &tok, buff, TOK_SHOW_COMMENTS );
+		 tok_has_next( &tok );
+		 tok_next( &tok ) )
+	{	
+		int last_type = tok_last_type( &tok );
+		int prev_argc=0;
+		
+		switch( last_type )
+		{
+			case TOK_STRING:
+			{
+				if( had_cmd )
+				{
+					
+					/*Parameter */
+					wchar_t *param = tok_last( &tok );
+					if( param[0] == L'-' )
+					{
+						if( complete_is_valid_option( last_cmd, param, error ))
+							color[ tok_get_pos( &tok ) ] = HIGHLIGHT_PARAM;
+						else
+							color[ tok_get_pos( &tok ) ] = HIGHLIGHT_ERROR;
+					}
+					else
+					{
+						color[ tok_get_pos( &tok ) ] = HIGHLIGHT_PARAM;
+					}					
+				}
+				else
+				{ 
+					prev_argc=0;
+					
+					/*
+					  Command. First check that the command actually exists.
+					*/
+					wchar_t *cmd = 
+						(last_type == TOK_STRING) ? 
+						expand_one(wcsdup(tok_last( &tok )),EXPAND_SKIP_SUBSHELL | EXPAND_SKIP_VARIABLES) : 
+						wcsdup(tok_last( &tok ));
+					if( cmd == 0 )
+					{
+						color[ tok_get_pos( &tok ) ] = HIGHLIGHT_ERROR;
+					}
+					else
+					{
+						wchar_t *tmp;
+						int is_cmd = 0;
+						int is_subcommand = 0;
+						int mark = tok_get_pos( &tok );
+						color[ tok_get_pos( &tok ) ] = HIGHLIGHT_COMMAND;
+
+						
+						if( parser_is_subcommand( cmd ) )
+						{
+							tok_next( &tok );
+							if(( wcscmp( L"-h", tok_last( &tok ) ) == 0 ) ||
+							   ( wcscmp( L"--help", tok_last( &tok ) ) == 0 ) )							
+							{
+								/* 
+								   The builtin and command builtins
+								   are normally followed by another
+								   command, but if they are invoked
+								   with the -h option, their help text
+								   is displayed instead
+								*/
+							}
+							else
+							{
+								is_subcommand = 1;
+							}
+							tok_set_pos( &tok, mark );
+						}
+
+						if( !is_subcommand )
+						{
+							/*
+							  OK, this is a command, it has been
+							  successfully expanded and everything
+							  looks ok. Lets check if the command
+							  exists.
+							*/
+							is_cmd |= builtin_exists( cmd );
+							is_cmd |= function_exists( cmd );
+							is_cmd |= (tmp=get_filename( cmd )) != 0;
+							
+							/* 
+							   Could not find the command. Maybe it is a path for a implicit cd command.
+							   Lets check!
+							*/
+							if( !is_cmd )
+							{
+								wchar_t *pp = parser_cdpath_get( cmd );
+								if( pp )
+								{
+									free( pp );
+									is_cmd = 1;
+								}		
+							}
+							
+							free(tmp);
+							if( is_cmd )
+							{								
+								color[ tok_get_pos( &tok ) ] = HIGHLIGHT_COMMAND;
+							}
+							else
+							{
+								if( error )
+									al_push( error, wcsdupcat2 ( L"Unknown command \'", cmd, L"\'", 0 ));
+								color[ tok_get_pos( &tok ) ] = (HIGHLIGHT_ERROR);
+							}
+							had_cmd = 1;
+						}
+						free(cmd);
+
+						if( had_cmd )
+						{
+							if( last_cmd )
+								free( last_cmd );
+							last_cmd = wcsdup( tok_last( &tok ) );						
+						}					
+
+					}
+
+				}
+				break;
+			}
+		
+			case TOK_REDIRECT_OUT:
+			case TOK_REDIRECT_IN:
+			case TOK_REDIRECT_APPEND:
+			case TOK_REDIRECT_FD:
+			{
+				if( !had_cmd )
+				{
+					color[ tok_get_pos( &tok ) ] = HIGHLIGHT_ERROR;
+					if( error )
+						al_push( error, wcsdup ( L"Redirection without a command" ) );
+					break;
+				}
+				
+				wchar_t *target=0;
+				
+				color[ tok_get_pos( &tok ) ] = HIGHLIGHT_REDIRECTION;
+				tok_next( &tok );
+
+				/*
+				  Check that we are redirecting into a file
+				*/
+
+				switch( tok_last_type( &tok ) )
+				{
+					case TOK_STRING:
+					{
+						target = expand_one( wcsdup( tok_last( &tok ) ), EXPAND_SKIP_SUBSHELL);
+						/*
+						  Redirect filename may contain a subshell. 
+						  If so, it will be ignored/not flagged.
+						*/
+					}
+					break;
+					default:
+					{
+						color[ tok_get_pos( &tok ) ] = HIGHLIGHT_ERROR;
+						if( error )
+							al_push( error, wcsdup ( L"Invalid redirection" ) );
+					}
+					
+				}
+
+				if( target != 0 )
+				{
+					wchar_t *dir = wcsdup( target );
+					wchar_t *dir_end = wcsrchr( dir, L'/' );
+					struct stat buff;
+					/* 
+					   If file is in directory other than '.', check
+					   that the directory exists.
+					*/
+					if( dir_end != 0 )
+					{
+						*dir_end = 0;
+						if( wstat( dir, &buff ) == -1 )
+						{
+							color[ tok_get_pos( &tok ) ] = HIGHLIGHT_ERROR;
+							if( error )
+								al_push( error, wcsdupcat2( L"Directory \'", dir, L"\' does not exist", 0 ) );
+							
+						}						
+					}
+					free( dir );
+					
+					/*
+					  If the file is read from or appended to, check
+					  if it exists.
+					*/
+					if( last_type == TOK_REDIRECT_IN || 
+						last_type == TOK_REDIRECT_APPEND )
+					{
+						if( wstat( target, &buff ) == -1 )
+						{
+							color[ tok_get_pos( &tok ) ] = HIGHLIGHT_ERROR;
+							if( error )
+								al_push( error, wcsdupcat2( L"File \'", target, L"\' does not exist", 0 ) );
+						}
+					}
+					free( target );					
+				}
+				break;
+			}
+			
+			case TOK_PIPE:
+			case TOK_BACKGROUND:
+			{
+				if( had_cmd )
+				{
+					color[ tok_get_pos( &tok ) ] = HIGHLIGHT_END;
+					had_cmd = 0;
+				}
+				else
+				{
+					color[ tok_get_pos( &tok ) ] = HIGHLIGHT_ERROR;					
+					if( error )
+						al_push( error, wcsdup ( L"No job to put in background" ) );
+				}
+				
+				break;
+			}
+			
+			case TOK_END:
+			{
+				color[ tok_get_pos( &tok ) ] = HIGHLIGHT_END;
+				had_cmd = 0;
+				break;
+			}
+			
+			case TOK_COMMENT:
+			{
+				color[ tok_get_pos( &tok ) ] = HIGHLIGHT_COMMENT;
+				break;
+			}
+			
+			case TOK_ERROR:
+			default:
+			{
+				/*
+				  If the tokenizer reports an error, highlight it as such.
+				*/
+				if( error )
+					al_push( error, wcsdup ( tok_last( &tok) ) );
+				color[ tok_get_pos( &tok ) ] = HIGHLIGHT_ERROR;
+				break;				
+			}			
+		}
+	}
+
+	if( last_cmd )
+		free( last_cmd );
+	
+	tok_destroy( &tok );	
+			 
+	/*
+	  Locate and syntax highlight subshells recursively
+	*/
+	
+	wchar_t *buffcpy = wcsdup( buff );
+	wchar_t *subpos=buffcpy;
+	int done=0;
+	
+	while( 1 )
+	{
+		wchar_t *begin, *end;
+		
+		if( expand_locate_subshell( subpos, 
+									&begin, 
+									&end,
+									1) <= 0)
+		{
+			break;
+		}
+		
+		if( !*end )
+			done=1;
+		else
+			*end=0;
+		
+		highlight_shell( begin+1, color +(begin-buffcpy)+1, -1, error );
+		color[end-buffcpy]=HIGHLIGHT_PARAM;
+		
+		if( done )
+			break;
+		
+		subpos = end+1;		
+	}
+	free( buffcpy );
+	
+
+	last_val=0;
+	for( i=0; buff[i] != 0; i++ )
+	{
+		if( color[i] >= 0 )
+			last_val = color[i];
+		else
+			color[i] = last_val;
+	}
+
+
+	highlight_universal_internal( buff, color, pos, error );
+
+	/*
+	  Spaces should not be highlighted at all, since it makes cursor look funky in some terminals
+	*/
+	for( i=0; buff[i]; i++ )
+	{
+		if( iswspace(buff[i]) )
+		{
+			color[i]=0;
+		}
+	}
+}
+
+/**
+   Perform quote and parenthesis highlighting on the specified string.
+*/
+static void highlight_universal_internal( wchar_t * buff, 
+										  int *color, 
+										  int pos, 
+										  array_list_t *error )
+{
+	
+	if( (pos >= 0) && (pos < wcslen(buff)) )
+	{
+		
+		/*
+		  Highlight matching quotes
+		*/
+		if( (buff[pos] == L'\'') || (buff[pos] == L'\"') )
+		{
+
+			array_list_t l;
+			al_init( &l );
+		
+			int level=0;
+			wchar_t prev_q=0;
+		
+			wchar_t *str=buff;
+
+			int match_found=0;
+		
+			while(*str)
+			{
+				switch( *str )
+				{
+					case L'\\':
+						str++;
+						break;
+					case L'\"':
+					case L'\'':
+						if( level == 0 )
+						{
+							level++;
+							al_push( &l, (void *)(str-buff) );
+							prev_q = *str;
+						}
+						else
+						{
+							if( prev_q == *str )
+							{
+								int pos1, pos2;
+							
+								level--;
+								pos1 = (int)al_pop( &l );
+								pos2 = str-buff;
+								if( pos1==pos || pos2==pos )
+								{
+									color[pos1]|=HIGHLIGHT_MATCH<<8;
+									color[pos2]|=HIGHLIGHT_MATCH<<8;
+									match_found = 1;
+									
+								}
+								prev_q = *str==L'\"'?L'\'':L'\"';
+							}
+							else
+							{
+								level++;
+								al_push( &l, (void *)(str-buff) );
+								prev_q = *str;
+							}
+						}
+					
+						break;
+				}
+				if( (*str == L'\0'))
+					break;
+
+				str++;
+			}
+
+			al_destroy( &l );
+		
+			if( !match_found )
+				color[pos] = HIGHLIGHT_ERROR<<8;
+		}
+
+		/*
+		  Highlight matching parenthesis
+		*/
+		if( wcschr( L"()[]{}", buff[pos] ) )
+		{
+			int step = wcschr(L"({[", buff[pos])?1:-1;
+			wchar_t dec_char = *(wcschr( L"()[]{}", buff[pos] ) + step);
+			wchar_t inc_char = buff[pos];
+			int level = 0;
+			wchar_t *str = &buff[pos];
+			int match_found=0;
+			
+
+			while( (str >= buff) && *str)
+			{
+				if( *str == inc_char )
+					level++;
+				if( *str == dec_char )
+					level--;
+				if( level == 0 )
+				{
+					int pos2 = str-buff;
+					color[pos]|=HIGHLIGHT_MATCH<<8;
+					color[pos2]|=HIGHLIGHT_MATCH<<8;
+					match_found=1;
+					break;
+				}
+				str+= step;
+			}
+			
+			if( !match_found )
+				color[pos] = HIGHLIGHT_ERROR<<8;
+		}
+	}
+}
+
+void highlight_universal( wchar_t * buff, 
+						  int *color, 
+						  int pos, 
+						  array_list_t *error )
+{
+	int i;
+	
+	for( i=0; buff[i] != 0; i++ )
+		color[i] = 0;
+	
+	highlight_universal_internal( buff, color, pos, error );
+}

highlight.h

@@ -0,0 +1,39 @@
+/** \file highlight.h
+	Prototypes for functions for syntax highlighting
+*/
+
+/**
+   Perform syntax highlighting for the shell commands in buff. The result is
+   stored in the color array as a color_code from the HIGHLIGHT_ enum
+   for each character in buff.
+
+   \param buff The buffer on which to perform syntax highlighting
+   \param color The array in wchich to store the color codes. The first 8 bits are used for fg color, the next 8 bits for bg color. 
+   \param pos the cursor position. Used for quote matching, etc.
+   \param error a list in which a description of each error will be inserted. May be 0, in whcich case no error descriptions will be generated.
+*/
+void highlight_shell( wchar_t * buff, int *color, int pos, array_list_t *error );
+
+/**
+   Perform syntax highlighting for the text in buff. Matching quotes and paranthesis are highlighted. The result is
+   stored in the color array as a color_code from the HIGHLIGHT_ enum
+   for each character in buff.
+
+   \param buff The buffer on which to perform syntax highlighting
+   \param color The array in wchich to store the color codes. The first 8 bits are used for fg color, the next 8 bits for bg color. 
+   \param pos the cursor position. Used for quote matching, etc.
+   \param error a list in which a description of each error will be inserted. May be 0, in whcich case no error descriptions will be generated.
+*/
+void highlight_universal( wchar_t * buff, int *color, int pos, array_list_t *error );
+
+/**
+   Translate from HIGHLIGHT_* to FISH_COLOR_* according to environment
+   variables. Defaults to FISH_COLOR_NORMAL.
+
+   Example: 
+
+   If the environment variable FISH_FISH_COLOR_ERROR is set to 'red', a
+   call to highlight_get_color( HIGHLIGHT_ERROR) will return
+   FISH_COLOR_RED.
+*/
+int highlight_get_color( int highlight );

history.c

@@ -0,0 +1,597 @@
+/** \file history.c
+  History functions, part of the user interface.
+*/
+#include <stdlib.h>
+#include <stdio.h>
+#include <wchar.h>
+#include <errno.h>
+#include <dirent.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+
+#include "config.h"
+#include "util.h"
+#include "wutil.h"
+#include "history.h"
+#include "common.h"
+#include "reader.h"
+#include "env.h"
+#include "sanity.h"
+
+/*
+  The history is implemented using a linked list. Searches are done
+  using linear search. 
+*/
+
+
+/**
+   A struct describing the state of the interactive history
+   list. Multiple states can be created. Use \c history_set_mode() to
+   change between history contexts.
+*/
+typedef struct
+{
+	/**
+	   Number of entries
+	*/
+	int count;
+	/**
+	   Last history item
+	*/
+	ll_node_t *last;
+	/**
+	   The last item loaded from file
+	*/
+	ll_node_t *last_loaded;
+}
+	history_data;
+
+
+static ll_node_t /** Last item in history */ *history_last=0, /** Current search position*/ *history_current =0;
+
+/**
+   We only try to save the part of the history that was written in
+   this session. This pointer keeps track of where the cutoff is.
+
+   The reason for wanting to do this is so that when two concurrent
+   fish sessions have exited, the latter one to exit won't push the
+   added history items of the prior session to the top of the history.
+*/
+static ll_node_t *last_loaded=0;
+
+/**
+  past_end is a kludge. It is set when the current history item is
+  unset, i.e. a new entry is being typed.
+*/
+static int past_end =1;
+
+/**
+   Items in history
+*/
+static int history_count=0;
+
+/**
+   The name of the current history list. The name is used to switch between history lists for different commands as sell as for deciding the name of the file to save the history in.
+*/
+static wchar_t *mode_name;
+
+/**
+   Hash table for storing all the history lists.
+*/
+static hash_table_t history_table;
+
+
+/**
+   Load history from ~/.fish_history
+*/
+static void history_load( wchar_t *name )
+{
+	wchar_t *fn;
+	wchar_t *buff=0;
+	int buff_len=0;
+	FILE *in_stream;
+	hash_table_t used;
+	
+	block();
+	hash_init2( &used, 
+				&hash_wcs_func,
+				&hash_wcs_cmp,
+				4096 );
+
+	fn = wcsdupcat2( env_get(L"HOME"), L"/.", name, L"_history", 0 );
+	
+	in_stream = wfopen( fn, "r" );
+	
+	if( in_stream != 0 )
+	{
+		while( !feof( in_stream ) )
+		{
+			int buff_read = fgetws2( &buff, &buff_len, in_stream );
+			if( buff_read == -1 )
+			{
+				debug( 1, L"The following non-fatal error occurred while reading command history from \'%ls\':", name );
+				wperror( L"fgetws2" );
+				fclose( in_stream );
+				free( fn );
+				free( buff );
+				unblock();				
+				return;
+			}
+
+			/*
+			  We do not call history_add here, since that would make
+			  history_load() take quadratic time, and may be
+			  unacceptably on slow systems with a long history file.
+
+			  Use a hashtable to check for duplicates instead.
+			*/
+			if( !hash_get( &used,
+						   buff ) )
+			{
+				history_count++;		
+				
+				history_current = malloc( sizeof( ll_node_t ) );
+				if( !history_current )
+				{
+					die_mem();
+					
+				}
+				
+				history_current->data=wcsdup( buff );
+
+				hash_put( &used,
+						  history_current->data, 
+						  L"" );
+				
+				history_current->next=0;
+				history_current->prev = history_last;
+				if( history_last != 0 )
+				{
+					history_last->next = history_current;
+				}
+				history_last = history_current;
+			}
+		}
+		
+		fclose( in_stream );
+	}
+	else
+	{
+		if( errno != ENOENT )
+		{
+			debug( 1, L"The following non-fatal error occurred while reading command history from \'%ls\':", name );
+			wperror( L"fopen" );
+		}
+		
+	}	
+
+	
+	hash_destroy( &used );
+	
+	free( buff );
+	free( fn );
+	last_loaded = history_last;
+	unblock();
+}
+
+void history_init()
+{
+	hash_init( &history_table,
+			   &hash_wcs_func,
+			   &hash_wcs_cmp );
+}
+
+/**
+   Make sure the current history list is in the history_table.
+*/
+static void history_to_hash()
+{
+	history_data *d;
+	
+	d = (history_data *)hash_get( &history_table, 
+								  mode_name );
+	
+	if( !d )
+		d = malloc( sizeof(history_data));
+	d->last=history_last;
+	d->last_loaded=last_loaded;
+	d->count=history_count;
+	
+	hash_put( &history_table, 
+			  mode_name,
+			  d );	
+	
+}
+
+
+void history_set_mode( wchar_t *name )
+{
+	history_data *curr;
+	
+	if( mode_name )
+	{		
+		history_to_hash();		
+	}
+	curr = (history_data *)hash_get( &history_table,
+									 name );
+	if( curr )
+	{
+		mode_name = (wchar_t *)hash_get_key( &history_table,
+											 name );
+		history_current = history_last = curr->last;
+		last_loaded = curr->last_loaded;
+		history_count = curr->count;
+	}
+	else
+	{
+		history_count=0;
+		history_last = history_current = last_loaded=0;
+		mode_name = wcsdup( name );
+		history_load( name );
+	}
+	
+	past_end=1;
+	
+}
+
+/**
+   Print history node 
+*/
+static void history_save_node( ll_node_t *n, FILE *out )
+{
+	if( n==0 )
+		return;
+	history_save_node( n->prev, out );
+	fwprintf(out, L"%ls\n", (wchar_t *)(n->data) );
+}
+
+/**
+   Merge the process history with the history on file, write to
+   disc. The merging operation is done so that two concurrently
+   running shells wont erase each others history.
+*/
+static void history_save()
+{
+	wchar_t *fn;
+	FILE *out_stream;
+	/* First we save this sessions history in local variables */
+	ll_node_t *real_pos = history_last, *real_first = last_loaded;
+
+	if( !real_first )
+	{
+		real_first = history_current;
+		while( real_first->prev )
+			real_first = real_first->prev;
+	}
+
+	if( real_pos == real_first )
+		return;
+	
+	/* Then we load the history from file into the global pointers */
+	history_last=history_current=last_loaded=0;
+	history_count=0;
+	past_end=1;
+
+	history_load( mode_name );
+	if( real_pos != 0 )
+	{
+		/* 
+		   Rewind the session history to the first item which was
+		   added in this session
+		*/
+		while( (real_pos->prev != 0) && (real_pos->prev != real_first) )
+		{
+			real_pos = real_pos->prev;
+		}
+		
+		/* Free old history entries */
+		ll_node_t *kill_node_t = real_pos->prev;
+		while( kill_node_t != 0 )
+		{
+			ll_node_t *tmp = kill_node_t;
+			free( kill_node_t->data );
+			kill_node_t = kill_node_t->prev;
+			free( tmp );		
+		}	
+
+		/* 
+		   Add all the history entries from this session to the global
+		   history, free the old version
+		*/
+		while( real_pos != 0 )
+		{
+			ll_node_t *next = real_pos->next;
+			history_add( (wchar_t *)real_pos->data );
+			
+			free( real_pos->data );
+			free( real_pos );
+			real_pos = next;
+		}
+	}
+
+	/* Save the global history */
+	{
+		fn = wcsdupcat2( env_get(L"HOME"), L"/.", mode_name, L"_history", 0 );
+		
+		out_stream = wfopen( fn, "w" );
+		if( out_stream )
+		{
+			history_save_node( history_last, out_stream );
+			if( fclose( out_stream ) )
+			{
+				debug( 1, L"The following non-fatal error occurred while saving command history to \'%ls\':", fn );
+				wperror( L"fopen" );
+			}			
+		}
+		else
+		{
+			debug( 1, L"The following non-fatal error occurred while saving command history to \'%ls\':", fn );
+			wperror( L"fopen" );
+		}
+		free( fn );	
+	}
+	
+}
+
+/**
+   Save the specified mode to file
+*/
+
+static void history_destroy_mode( const void *name, const void *link )
+{
+	mode_name = (wchar_t *)name;
+	history_data *d = (history_data *)link;
+	history_last = history_current = d->last;
+	last_loaded = d->last_loaded;
+	history_count = d->count;
+	past_end=1;
+
+//	fwprintf( stderr, L"Destroy history mode \'%ls\'\n", mode_name );
+		
+	if( history_last )
+	{
+		history_save();	
+		history_current = history_last;
+		while( history_current != 0 )
+		{
+			ll_node_t *tmp = history_current;
+			free( history_current->data );
+			history_current = history_current->prev;
+			free( tmp );		
+		}	
+	}
+	free( d );
+	free( mode_name );
+}
+
+void history_destroy()
+{
+	/**
+	   Make sure current mode is in table
+	*/
+	history_to_hash();		
+
+	/**
+	   Save all modes in table
+	*/
+	hash_foreach( &history_table, 
+				  &history_destroy_mode );
+
+	hash_destroy( &history_table );
+	
+}
+
+
+
+/**
+   Internal search function
+*/
+static ll_node_t *history_find( ll_node_t *n, const wchar_t *s )
+{
+	if( n == 0 )
+		return 0;
+	
+	if( wcscmp( s, (wchar_t *)n->data ) == 0 )
+	{
+		return n;		
+	}
+	else
+		return history_find( n->prev, s );
+}
+
+
+void history_add( const wchar_t *str )
+{
+	ll_node_t *old_node;
+
+	if( wcslen( str ) == 0 )
+		return;
+	
+	past_end=1;
+
+	old_node = history_find( history_last, str );
+
+	if( old_node == 0 )
+	{
+		history_count++;		
+		history_current = malloc( sizeof( ll_node_t ) );
+		history_current->data=wcsdup( str );
+	}
+	else
+	{
+		if( old_node == last_loaded )
+		{
+			last_loaded = last_loaded->prev;
+		}
+	
+		history_current = old_node;
+		if( old_node == history_last )
+		{
+			return;
+		}
+
+		if( old_node->prev != 0 )
+			old_node->prev->next = old_node->next;
+		if( old_node->next != 0 )
+			old_node->next->prev = old_node->prev;
+	}
+	history_current->next=0;
+	history_current->prev = history_last;
+	if( history_last != 0 )
+	{
+		history_last->next = history_current;
+	}
+	history_last = history_current;
+}
+
+/**
+   This function tests if the search string is a match for the given string
+*/
+static int history_test( const wchar_t *needle, const wchar_t *haystack )
+{
+
+/*
+	return wcsncmp( haystack, needle, wcslen(needle) )==0;
+*/
+	return (int)wcsstr( haystack, needle );
+}
+
+const wchar_t *history_prev_match( const wchar_t *str )
+{
+	if( history_current == 0 )
+		return str;
+	
+	if( history_current->prev == 0 )
+	{
+		if( history_test( str, history_current->data ) )
+			return (wchar_t *)history_current->data;
+		else
+			return str;
+	}
+	if( past_end )
+		past_end = 0;
+	else
+		history_current = history_current->prev;
+
+	if( history_test( str, history_current->data) )
+		return (wchar_t *)history_current->data;
+	else
+		return history_prev_match( str );
+}
+
+
+const wchar_t *history_next_match( const wchar_t *str)
+{
+	if( history_current == 0 )
+		return str;
+
+	if( history_current->next == 0 )
+	{
+		past_end = 1;
+		return str;
+	}
+	history_current = history_current->next;
+	
+	if( history_test( str, history_current->data ) )
+		return (wchar_t *)history_current->data;
+	else
+		return history_next_match( str );	
+}
+
+void history_reset()
+{
+	history_current = history_last;
+}
+
+/**
+   Move to first history item
+*/
+void history_first()
+{
+	while( history_current->prev )
+		history_current = history_current->prev;
+	
+}
+
+wchar_t *history_get( int idx )
+{
+	ll_node_t *n=history_last;
+	int i;
+	if( idx<0)
+	{
+		debug( 1, L"Tried to access negative history index %d", idx );
+		return 0;
+	}
+	
+	for( i=0; i<idx; i++ )
+	{
+		if( !n )
+			break;
+		n=n->prev;
+	}
+	return n?n->data:0;
+}
+
+
+void history_sanity_check()
+{	
+	return;
+	
+	if( history_current != 0 )
+	{
+		int i;
+		int history_ok = 1;
+		int found_current = 0;
+		
+		validate_pointer( history_last, L"History root", 1);
+		
+		ll_node_t *tmp = history_last;
+		for( i=0; i<history_count; i++ )
+		{
+			found_current |= tmp == history_current;
+
+			if( tmp == 0 )
+			{
+				history_ok = 0;
+				debug( 1, L"History items missing" );
+				break;
+			}
+
+			if( (tmp->prev != 0) && (tmp->prev->next != tmp ) )
+			{
+				history_ok = 0;
+				debug( 1, L"History items order is inconsistent" );
+				break;
+			}
+			
+			validate_pointer( tmp->data, L"History data", 1);
+			if( tmp->data == 0 )
+			{
+				history_ok = 0;
+				debug( 1, L"Empty history item" );
+				break;
+			}
+			validate_pointer( tmp->prev, L"History node", 1);
+			tmp = tmp->prev;
+		}
+		if( tmp != 0 )
+		{
+			history_ok = 0;
+			debug( 1, L"History list too long" );
+		}
+
+		if( (i!= history_count )|| (!found_current))
+		{
+			debug( 1, L"No history item selected" );
+			history_ok=0;
+		}
+		
+		if( !history_ok )
+		{
+			sanity_lose();			
+		}
+	}	
+}
+

history.h

@@ -0,0 +1,65 @@
+/** \file history.h
+  Prototypes for history functions, part of the user interface.
+*/
+
+/**
+   Load history from file.
+*/
+void history_init();
+
+/**
+   Saves the new history to disc and frees all memory used by the history.
+*/
+void history_destroy();
+
+/**
+  Add a new history item to the bottom of the history, containing a
+  copy of str. Remove any duplicates. Moves the current item past the
+  end of the history list.
+*/
+void history_add( const wchar_t *str );
+
+/**
+  Find previous history item starting with str. If this moves before
+  the start of the history, str is returned.
+*/
+const wchar_t *history_prev_match( const wchar_t *str );
+
+/**
+   Return the specified history at the specified index, or 0 if out of bounds. 0 is the index of the current commandline.
+*/
+wchar_t *history_get( int idx );
+
+
+/**
+   Move to first history item
+*/
+void history_first();
+
+/**
+  Make current point to last history item
+*/
+void history_reset();
+
+
+/**
+  Find next history item starting with str. If this moves past
+  the end of the history, str is returned.  
+*/
+const wchar_t *history_next_match( const wchar_t *str);
+
+
+/**
+   Set the current mode name for history. Each application that uses
+   the history has it's own mode. This must be called prior to any use
+   of the history.
+*/
+
+void history_set_mode( wchar_t *name );
+
+
+/**
+   Perform sanity checks
+*/
+void history_sanity_check();
+

input.h

@@ -0,0 +1,109 @@
+/** \file input.h
+
+Functions for reading a character of input from stdin, using the
+inputrc information for key bindings.
+
+*/
+
+/**
+   Key codes for inputrc-style keyboard functions that are passed on
+   to the caller of input_read()
+*/
+enum
+{
+	R_BEGINNING_OF_LINE = R_NULL+1, 
+	R_END_OF_LINE,
+	R_FORWARD_CHAR,
+	R_BACKWARD_CHAR,
+	R_FORWARD_WORD,
+	R_BACKWARD_WORD,
+	R_HISTORY_SEARCH_BACKWARD,
+	R_HISTORY_SEARCH_FORWARD,
+	R_DELETE_CHAR,
+	R_BACKWARD_DELETE_CHAR,
+	R_KILL_LINE,
+	R_YANK,
+	R_YANK_POP,
+	R_COMPLETE,
+	R_BEGINNING_OF_HISTORY,
+	R_END_OF_HISTORY,
+	R_DELETE_LINE,
+	R_BACKWARD_KILL_LINE,
+	R_KILL_WHOLE_LINE,
+	R_KILL_WORD,
+	R_BACKWARD_KILL_WORD,
+	R_DUMP_FUNCTIONS,
+	R_CLEAR_SCREEN,
+	R_EXIT,
+	R_HISTORY_TOKEN_SEARCH_BACKWARD,
+	R_HISTORY_TOKEN_SEARCH_FORWARD,
+	R_SELF_INSERT,
+}
+;
+
+/**
+   Initialize the terminal by calling setupterm, and set up arrays
+   used by readch to detect escape sequences for special keys.
+
+   Before calling input_init, terminfo is not initialized and MUST not be used
+*/
+int input_init();
+
+/**
+   free up memory used by terminal functions.
+*/
+void input_destroy();
+
+/**
+   Read a character from fd 0. Try to convert some escape sequences
+   into character constants, but do not permanently block the escape
+   character.
+
+   This is performed in the same way vim does it, i.e. if an escape
+   character is read, wait for more input for a short time (a few
+   milliseconds). If more input is avaialable, it is assumed to be an
+   escape sequence for a special character (such as an arrow key), and
+   readch attempts to parse it. If no more input follows after the
+   escape key, it is assumed to be an actual escape key press, and is
+   returned as such.
+*/
+wint_t input_readch();
+
+/**
+   Push a character or a readline function onto the stack of unread
+   characters that input_readch will return before actually reading from fd
+   0.
+*/
+void input_unreadch( wint_t ch );
+
+
+/**
+   Add a key mapping from the specified sequence
+
+   \param mode the name of the mapping mode to add this mapping to
+   \param s the sequence
+   \param d a description of the sequence
+   \param c am input function that will be run whenever the key sequence occurs
+*/
+void add_mapping( const wchar_t *mode, const wchar_t *s, const wchar_t * d, const wchar_t *cmd );
+
+/**
+   Sets the mode keybindings. 
+*/
+void input_set_mode( wchar_t *name );
+
+/**
+   Sets the application keybindings
+*/
+void input_set_application( wchar_t *name );
+
+/**
+   Parse a single line of inputrc information. 
+*/
+void input_parse_inputrc_line( wchar_t *cmd );
+
+/**
+   Returns the function for the given function name.
+*/
+wchar_t input_get_code( wchar_t *name );
+

input_common.c

@@ -0,0 +1,220 @@
+/** \file input_common.h
+	
+Implementation file for the low level input library
+
+*/
+#include "config.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <unistd.h>
+#include <wchar.h>
+
+#include "util.h"
+#include "common.h"
+#include "wutil.h"
+#include "input_common.h"
+#include "env_universal.h"
+
+/**
+   Time in milliseconds to wait for another byte to be available for
+   reading after \e is read before assuming that escape key was
+   pressed, and not an escape sequence.
+*/
+#define WAIT_ON_ESCAPE 10
+
+/**
+   Characters that have been read and returned by the sequence matching code
+*/
+static wint_t lookahead_arr[32];
+
+/**
+   Number of entries in lookahead_arr
+*/
+static int lookahead_count = 0;
+
+/**
+   Callback function for handling interrupts on reading
+*/
+static int (*interrupt_handler)();
+
+void input_common_init( int (*ih)() )
+{
+	interrupt_handler = ih;
+}
+
+void input_common_destroy()
+{
+	
+}
+
+/**
+   Internal function used by input_common_readch to read one byte from fd 1. This function should only be called by
+   input_common_readch().
+*/
+static wint_t readb()
+{
+	char arr[1];
+	int do_loop = 0;
+
+	do
+	{
+		fd_set fd;	
+		int fd_max=1;
+		int res;
+		
+		FD_ZERO( &fd );
+		FD_SET( 0, &fd );
+		if( env_universal_server.fd > 0 )
+		{
+			FD_SET( env_universal_server.fd, &fd );
+			fd_max = env_universal_server.fd+1;
+		}
+		
+		do_loop = 0;			
+		
+		res = select( fd_max, &fd, 0, 0, 0 );
+		if( res==-1 )
+		{
+			switch( errno )
+			{
+				case EINTR:
+				case EAGAIN:
+				{
+//					wperror( L"select" );
+					if( interrupt_handler )
+					{
+						int res = interrupt_handler();
+						
+/*						debug( 0,
+							   L"interrupt, %d is %ls", 
+							   res, 
+							   (res==R_NULL?L"good": L"Bad") );
+*/
+						if( res )
+							return res;
+					}
+					
+					do_loop = 1;
+					break;
+				}
+				default:
+				{
+					debug( 0, L"Error while reading input from keyboard, shutting down" );
+					wperror(L"read");
+					exit(1);
+				}
+			}	
+		}
+		else
+		{
+			if( env_universal_server.fd > 0 )
+			{
+				if( FD_ISSET( env_universal_server.fd, &fd ) )
+				{
+					debug( 3, L"Wake up on universal variable event" );					
+					env_universal_read_all();
+					return R_NULL;
+				}				
+			}
+			if( FD_ISSET( 0, &fd ) )
+			{
+				if( read_blocked( 0, arr, 1 ) == -1 )
+				{
+					debug( 0, L"Error while reading input from keyboard, shutting down" );
+					wperror(L"read");
+					exit(1);
+				}
+				do_loop = 0;
+			}				
+		}
+	}
+	while( do_loop );
+	
+	return arr[0];
+}
+
+wchar_t input_common_readch( int timed )
+{
+	if( lookahead_count == 0 )
+	{
+		if( timed )
+		{
+			int count;
+			fd_set fds;
+			struct timeval tm=
+				{
+					0,
+					1000 * WAIT_ON_ESCAPE
+				}
+			;
+			
+			FD_ZERO( &fds );
+			FD_SET( 0, &fds );
+			count = select(1, &fds, 0, 0, &tm);
+			
+			switch( count )
+			{
+				case 0:
+					return WEOF;
+					
+				case -1:
+					return WEOF;
+					break;
+				default:
+					break;
+
+			}
+		}
+		
+		wchar_t res;
+		static mbstate_t state;
+
+		while(1)
+		{
+			wint_t b = readb();
+			int sz;
+			
+			if( b == R_NULL )
+				return R_NULL;
+
+			sz = mbrtowc( &res, &b, 1, &state );
+			
+			switch( sz )
+			{
+				case -1:
+					memset (&state, '\0', sizeof (state));
+					debug( 2, L"Illegal input" );					
+					return R_NULL;					
+				case -2:
+					break;
+				case 0:
+					return 0;
+				default:
+					
+					return res;
+			}
+		}
+	}
+	else 
+	{
+		if( !timed )
+		{
+			while( (lookahead_count >= 0) && (lookahead_arr[lookahead_count-1] == WEOF) )
+				lookahead_count--;
+			if( lookahead_count == 0 )
+				return input_common_readch(0);
+		}
+		
+		return lookahead_arr[--lookahead_count];
+	}
+}
+
+
+void input_common_unreadch( wint_t ch )
+{
+	lookahead_arr[lookahead_count++] = ch;
+}
+

input_common.h

@@ -0,0 +1,40 @@
+/** \file input_common.h
+	
+Header file for the low level input library
+
+*/
+#ifndef INPUT_COMMON_HH
+#define INPUT_COMMON_HH
+
+/** 
+	Hopefully, the biggest value that a wchar_t can have. UCS4 is a
+	31-bit character set, we use the upper half for special key
+	sequences. On systems where wchar_t is not a 31 (or less) bit character set
+	in a 32 (or more) bit type this will fail horribly. 
+*/
+#define WCHAR_END 0x80000000
+
+enum
+{
+	R_NULL = WCHAR_END + 1
+}
+	;
+
+void input_common_init( int (*ih)() );
+
+void input_common_destroy();
+
+/**
+   Function used by input_readch to read bytes from stdin until enough
+   bytes have been read to convert them to a wchar_t. Conversion is
+   done using mbrtowc. If a character has previously been read and
+   then 'unread' using \c input_common_unreadch, that character is
+   returned. If timed is true, readch2 will wait at most
+   WAIT_ON_ESCAPE milliseconds for a character to be available for
+   reading before returning with the value WEOF.
+*/
+wchar_t input_common_readch( int timed );
+
+void input_common_unreadch( wint_t ch );
+
+#endif

install-sh

@@ -0,0 +1,251 @@
+#!/bin/sh
+#
+# install - install a program, script, or datafile
+# This comes from X11R5 (mit/util/scripts/install.sh).
+#
+# Copyright 1991 by the Massachusetts Institute of Technology
+#
+# Permission to use, copy, modify, distribute, and sell this software and its
+# documentation for any purpose is hereby granted without fee, provided that
+# the above copyright notice appear in all copies and that both that
+# copyright notice and this permission notice appear in supporting
+# documentation, and that the name of M.I.T. not be used in advertising or
+# publicity pertaining to distribution of the software without specific,
+# written prior permission.  M.I.T. makes no representations about the
+# suitability of this software for any purpose.  It is provided "as is"
+# without express or implied warranty.
+#
+# Calling this script install-sh is preferred over install.sh, to prevent
+# `make' implicit rules from creating a file called install from it
+# when there is no Makefile.
+#
+# This script is compatible with the BSD install script, but was written
+# from scratch.  It can only install one file at a time, a restriction
+# shared with many OS's install programs.
+
+
+# set DOITPROG to echo to test this script
+
+# Don't use :- since 4.3BSD and earlier shells don't like it.
+doit="${DOITPROG-}"
+
+
+# put in absolute paths if you don't have them in your path; or use env. vars.
+
+mvprog="${MVPROG-mv}"
+cpprog="${CPPROG-cp}"
+chmodprog="${CHMODPROG-chmod}"
+chownprog="${CHOWNPROG-chown}"
+chgrpprog="${CHGRPPROG-chgrp}"
+stripprog="${STRIPPROG-strip}"
+rmprog="${RMPROG-rm}"
+mkdirprog="${MKDIRPROG-mkdir}"
+
+transformbasename=""
+transform_arg=""
+instcmd="$mvprog"
+chmodcmd="$chmodprog 0755"
+chowncmd=""
+chgrpcmd=""
+stripcmd=""
+rmcmd="$rmprog -f"
+mvcmd="$mvprog"
+src=""
+dst=""
+dir_arg=""
+
+while [ x"$1" != x ]; do
+    case $1 in
+	-c) instcmd="$cpprog"
+	    shift
+	    continue;;
+
+	-d) dir_arg=true
+	    shift
+	    continue;;
+
+	-m) chmodcmd="$chmodprog $2"
+	    shift
+	    shift
+	    continue;;
+
+	-o) chowncmd="$chownprog $2"
+	    shift
+	    shift
+	    continue;;
+
+	-g) chgrpcmd="$chgrpprog $2"
+	    shift
+	    shift
+	    continue;;
+
+	-s) stripcmd="$stripprog"
+	    shift
+	    continue;;
+
+	-t=*) transformarg=`echo $1 | sed 's/-t=//'`
+	    shift
+	    continue;;
+
+	-b=*) transformbasename=`echo $1 | sed 's/-b=//'`
+	    shift
+	    continue;;
+
+	*)  if [ x"$src" = x ]
+	    then
+		src=$1
+	    else
+		# this colon is to work around a 386BSD /bin/sh bug
+		:
+		dst=$1
+	    fi
+	    shift
+	    continue;;
+    esac
+done
+
+if [ x"$src" = x ]
+then
+	echo "install:	no input file specified"
+	exit 1
+else
+	true
+fi
+
+if [ x"$dir_arg" != x ]; then
+	dst=$src
+	src=""
+	
+	if [ -d $dst ]; then
+		instcmd=:
+		chmodcmd=""
+	else
+		instcmd=mkdir
+	fi
+else
+
+# Waiting for this to be detected by the "$instcmd $src $dsttmp" command
+# might cause directories to be created, which would be especially bad 
+# if $src (and thus $dsttmp) contains '*'.
+
+	if [ -f $src -o -d $src ]
+	then
+		true
+	else
+		echo "install:  $src does not exist"
+		exit 1
+	fi
+	
+	if [ x"$dst" = x ]
+	then
+		echo "install:	no destination specified"
+		exit 1
+	else
+		true
+	fi
+
+# If destination is a directory, append the input filename; if your system
+# does not like double slashes in filenames, you may need to add some logic
+
+	if [ -d $dst ]
+	then
+		dst="$dst"/`basename $src`
+	else
+		true
+	fi
+fi
+
+## this sed command emulates the dirname command
+dstdir=`echo $dst | sed -e 's,[^/]*$,,;s,/$,,;s,^$,.,'`
+
+# Make sure that the destination directory exists.
+#  this part is taken from Noah Friedman's mkinstalldirs script
+
+# Skip lots of stat calls in the usual case.
+if [ ! -d "$dstdir" ]; then
+defaultIFS='	
+'
+IFS="${IFS-${defaultIFS}}"
+
+oIFS="${IFS}"
+# Some sh's can't handle IFS=/ for some reason.
+IFS='%'
+set - `echo ${dstdir} | sed -e 's@/@%@g' -e 's@^%@/@'`
+IFS="${oIFS}"
+
+pathcomp=''
+
+while [ $# -ne 0 ] ; do
+	pathcomp="${pathcomp}${1}"
+	shift
+
+	if [ ! -d "${pathcomp}" ] ;
+        then
+		$mkdirprog "${pathcomp}"
+	else
+		true
+	fi
+
+	pathcomp="${pathcomp}/"
+done
+fi
+
+if [ x"$dir_arg" != x ]
+then
+	$doit $instcmd $dst &&
+
+	if [ x"$chowncmd" != x ]; then $doit $chowncmd $dst; else true ; fi &&
+	if [ x"$chgrpcmd" != x ]; then $doit $chgrpcmd $dst; else true ; fi &&
+	if [ x"$stripcmd" != x ]; then $doit $stripcmd $dst; else true ; fi &&
+	if [ x"$chmodcmd" != x ]; then $doit $chmodcmd $dst; else true ; fi
+else
+
+# If we're going to rename the final executable, determine the name now.
+
+	if [ x"$transformarg" = x ] 
+	then
+		dstfile=`basename $dst`
+	else
+		dstfile=`basename $dst $transformbasename | 
+			sed $transformarg`$transformbasename
+	fi
+
+# don't allow the sed command to completely eliminate the filename
+
+	if [ x"$dstfile" = x ] 
+	then
+		dstfile=`basename $dst`
+	else
+		true
+	fi
+
+# Make a temp file name in the proper directory.
+
+	dsttmp=$dstdir/#inst.$$#
+
+# Move or copy the file name to the temp name
+
+	$doit $instcmd $src $dsttmp &&
+
+	trap "rm -f ${dsttmp}" 0 &&
+
+# and set any options; do chmod last to preserve setuid bits
+
+# If any of these fail, we abort the whole thing.  If we want to
+# ignore errors from any of these, just make sure not to ignore
+# errors from the above "$doit $instcmd $src $dsttmp" command.
+
+	if [ x"$chowncmd" != x ]; then $doit $chowncmd $dsttmp; else true;fi &&
+	if [ x"$chgrpcmd" != x ]; then $doit $chgrpcmd $dsttmp; else true;fi &&
+	if [ x"$stripcmd" != x ]; then $doit $stripcmd $dsttmp; else true;fi &&
+	if [ x"$chmodcmd" != x ]; then $doit $chmodcmd $dsttmp; else true;fi &&
+
+# Now rename the file to the real destination.
+
+	$doit $rmcmd -f $dstdir/$dstfile &&
+	$doit $mvcmd $dsttmp $dstdir/$dstfile 
+
+fi &&
+
+
+exit 0

intern.c

@@ -0,0 +1,119 @@
+/** \file intern.c
+
+    Library for pooling common strings
+
+*/
+#include "config.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <wchar.h>
+
+#include "util.h"
+#include "common.h"
+#include "intern.h"
+
+hash_table_t *intern_table=0;
+hash_table_t *intern_static_table=0;
+
+static void intern_load_common_static()
+{
+	intern_static( L"" );
+}
+
+const wchar_t *intern( const wchar_t *in )
+{
+	const wchar_t *res=0;
+	
+	if( !in )
+		return 0;
+	
+	intern_load_common_static();
+	
+
+	if( !intern_table )
+	{
+		intern_table = malloc( sizeof( hash_table_t ) );
+		if( !intern_table )
+		{
+			die_mem();
+		}
+		hash_init( intern_table, &hash_wcs_func, &hash_wcs_cmp );
+	}
+	
+	if( intern_static_table )
+	{
+		res = hash_get( intern_static_table, in );
+	}
+	
+	if( !res )
+	{
+		res = hash_get( intern_table, in );
+		
+		if( !res )
+		{
+			res = wcsdup( in );
+			if( !res )
+			{
+				die_mem();
+			}
+			
+			hash_put( intern_table, res, res );
+		}
+	}
+	
+	return res;
+}
+
+const wchar_t *intern_static( const wchar_t *in )
+{
+	const wchar_t *res=0;
+	
+	if( !in )
+		return 0;
+	
+	if( !intern_static_table )
+	{
+		intern_static_table = malloc( sizeof( hash_table_t ) );
+		if( !intern_static_table )
+		{
+			die_mem();			
+		}
+		hash_init( intern_static_table, &hash_wcs_func, &hash_wcs_cmp );
+	}
+	
+	res = hash_get( intern_static_table, in );
+	
+	if( !res )
+	{
+		res = in;
+		hash_put( intern_static_table, res, res );
+	}
+	
+	return res;
+}
+
+static void clear_value( const void *key, const void *data )
+{
+	debug( 3,  L"interned string: '%ls'", data );	
+	free( (void *)data );
+}
+
+void intern_free_all()
+{
+	if( intern_table )
+	{
+		hash_foreach( intern_table, &clear_value );		
+		hash_destroy( intern_table );		
+		free( intern_table );
+		intern_table=0;		
+	}
+
+	if( intern_static_table )
+	{
+		hash_destroy( intern_static_table );		
+		free( intern_static_table );
+		intern_static_table=0;		
+	}
+	
+}

intern.h

@@ -0,0 +1,24 @@
+/** \file intern.h
+
+    Library for pooling common strings
+
+*/
+
+/**
+   Return an identical copy of the specified string from a pool of unique strings. If the string was not in the pool, add a copy.
+
+   \param The string to return an interned copy of
+*/
+const wchar_t *intern( const wchar_t *in );
+
+/**
+   Insert the specified string literal into the pool of unique
+   strings. The string will not first be copied, and it will not be
+   free'd on exit.
+*/
+const wchar_t *intern_static( const wchar_t *in );
+
+/**
+   Free all interned strings
+*/
+void intern_free_all();

key_reader.c

@@ -0,0 +1,92 @@
+/*
+    A small utility to print the resulting key codes from pressing a
+	key. Servers the same function as hitting ^V in bash, but I prefer
+	the way key_reader works.
+
+	Type ^C to exit the program.
+*/
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <termios.h>
+#include <unistd.h>
+#include <termcap.h>
+
+static int readch()
+{
+	char arr[1];
+	if( read( 0, arr, 1 ) < 0 )
+	{
+		perror( "read" );
+		return readch();		
+	}
+	else
+		return arr[0];
+}
+
+int writestr( char *str )
+{
+	write( 1, str, strlen(str) );
+	return 0;
+}
+
+int main( int argc, char **argv)
+{
+
+	if( argc == 2 )
+	{
+		static char term_buffer[2048];
+		char *termtype = getenv ("TERM");
+		char *tbuff = malloc( sizeof(char)*9999);
+		char *res;
+		
+		tgetent( term_buffer, termtype );
+		res = tgetstr( argv[1], &tbuff );		
+		if( res != 0 )
+		{
+			while( *res != 0 )
+			{ 
+				printf("%d ", *res );
+
+
+				res++;
+			}
+			printf( "\n" );		
+		}
+		else
+		{
+			printf("Undefined sequence\n");
+		}		
+	}
+	else
+	{
+		char scratch[1024];
+		unsigned int c;
+
+		struct termios modes,      /* so we can change the modes */
+			savemodes;  /* so we can reset the modes when we're done */
+
+		tcgetattr(0,&modes);        /* get the current terminal modes */
+		savemodes = modes;          /* save a copy so we can reset them */
+		
+		modes.c_lflag &= ~ICANON;   /* turn off canonical mode */
+		modes.c_lflag &= ~ECHO;   /* turn off echo mode */
+		modes.c_cc[VMIN]=1;
+		modes.c_cc[VTIME]=0;
+		tcsetattr(0,TCSANOW,&modes);      /* set the new modes */
+		while(1)
+		{
+			if( (c=readch()) == EOF )
+				break;
+			if((c > 31) && (c != 127) )
+				sprintf( scratch, "dec: %d hex: %x char: %c\n", c, c, c );
+			else
+				sprintf( scratch, "dec: %d hex: %x\n", c, c );
+			writestr( scratch );			
+		}
+		/* reset the terminal to the saved mode */
+		tcsetattr(0,TCSANOW,&savemodes);  
+	}	
+
+	return 0;
+}

kill.c

@@ -0,0 +1,256 @@
+/** \file kill.c
+	The killring.
+
+	Works like the killring in emacs and readline. The killring is cut
+	and paste with a memory of previous cuts. It supports integration
+	with the X clipboard.
+*/
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <wchar.h>
+#include <termios.h>
+#include <unistd.h>
+#include <signal.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <dirent.h>
+
+
+#include "config.h"
+#include "util.h"
+#include "wutil.h"
+#include "kill.h"
+#include "proc.h"
+#include "sanity.h"
+#include "common.h"
+#include "env.h"
+#include "expand.h"
+#include "exec.h"
+#include "parser.h"
+
+/**
+   Maximum entries in killring
+*/
+#define KILL_MAX 8192
+
+
+static ll_node_t /** Last kill string */*kill_last=0, /** Current kill string */*kill_current =0;
+/**
+   Contents of the X clipboard, at last time we checked it
+*/
+static wchar_t *cut_buffer=0;
+
+/**
+   Test if the xsel command is installed
+*/
+static int has_xsel()
+{
+	wchar_t *path = get_filename( L"xsel" );
+	if( path)
+	{
+		free(path);
+		return 1;
+	}
+	else
+		return 0;
+}
+
+
+
+/**
+   Add the string to the internal killring
+*/
+static void kill_add_internal( wchar_t *str )
+{
+	if( wcslen( str ) == 0 )
+		return;
+	
+	if( kill_last == 0 )
+	{
+		kill_current = kill_last=malloc( sizeof( ll_node_t ) );
+		kill_current->data = wcsdup(str);
+		kill_current->prev = kill_current;
+	}
+	else
+	{
+		kill_current = malloc( sizeof( ll_node_t ) );
+		kill_current->data = kill_last->data;
+		kill_last->data = wcsdup(str);
+		kill_current->prev = kill_last->prev;
+		kill_last->prev = kill_current;
+		kill_current = kill_last;
+	}
+}
+
+
+void kill_add( wchar_t *str )
+{
+	kill_add_internal(str);
+
+	if( !has_xsel() )
+		return;
+
+	/* This is for sending the kill to the X copy-and-paste buffer */
+	wchar_t *disp;
+	if( (disp = env_get( L"DISPLAY" )) )
+	{
+		wchar_t *escaped_str = expand_escape( wcsdup(str), 1 );
+		wchar_t *cmd = wcsdupcat2(L"echo ", escaped_str, L"|xsel -b",0);
+		exec_subshell( cmd, 0 );
+		free( cut_buffer );
+		free( cmd );
+	
+		cut_buffer = escaped_str;
+	}
+}
+
+
+wchar_t *kill_yank_rotate()
+{
+	if( kill_current == 0 )
+		return L"";
+	kill_current = kill_current->prev;
+	return (wchar_t *)kill_current->data;
+}
+
+/**
+   Check the X clipboard. If it has been changed, add the new
+   clipboard contents to the fish killring.
+*/
+static void kill_check_x_buffer()
+{	
+	wchar_t *disp;
+	
+	if( !has_xsel() )
+		return;
+	
+	
+	if( (disp = env_get( L"DISPLAY" )) )
+	{
+		int i;
+		wchar_t *cmd = L"xsel -t 500 -b";
+		wchar_t *new_cut_buffer=0;
+		array_list_t list;
+		al_init( &list );
+		exec_subshell( cmd, &list );
+		
+		for( i=0; i<al_get_count( &list ); i++ )
+		{
+			wchar_t *next_line = expand_escape(wcsdup(al_get( &list, i )), 0);
+			if( i==0 )
+			{
+				new_cut_buffer = next_line;
+			}
+			else
+			{
+				wchar_t *old = new_cut_buffer;
+				new_cut_buffer= wcsdupcat2( new_cut_buffer, L"\\n", next_line, 0 );
+				free( old );
+				free( next_line );	
+			}
+		}
+		
+		if( new_cut_buffer )
+		{
+			/*  
+				The buffer is inserted with backslash escapes,
+				since we don't really like tabs, newlines,
+				etc. anyway.
+			*/
+			
+			if( cut_buffer != 0 )
+			{      
+				if( wcscmp( new_cut_buffer, cut_buffer ) == 0 )
+				{
+					free( new_cut_buffer );
+					new_cut_buffer = 0;
+				}
+				else
+				{
+					free( cut_buffer );
+					cut_buffer = 0;								
+				}
+			}
+			if( cut_buffer == 0 )
+			{
+				cut_buffer = new_cut_buffer;
+				kill_add_internal( cut_buffer );
+			}
+		}
+		
+		al_foreach( &list, (void (*)(const void *))&free );
+		al_destroy( &list );
+	}
+}
+
+
+wchar_t *kill_yank()
+{
+	kill_check_x_buffer();
+	if( kill_current == 0 )
+		return L"";
+	kill_current=kill_last;
+	return (wchar_t *)kill_current->data;
+}
+
+void kill_sanity_check()
+{
+	int i;
+	if( is_interactive )
+	{
+		/* Test that the kill-ring is consistent */
+		if( kill_current != 0 )
+		{
+			int kill_ok = 0;
+			ll_node_t *tmp = kill_current->prev;
+			for( i=0; i<KILL_MAX; i++ )
+			{
+				if( tmp == 0 )
+					break;
+				if( tmp->data == 0 )
+					break;
+				
+				if( tmp == kill_current )
+				{
+					kill_ok = 1;
+					break;
+				}
+				tmp = tmp->prev;				
+			}
+			if( !kill_ok )
+			{
+				debug( 0, 
+					   L"Killring inconsistent" );
+				sanity_lose();
+			}
+		}
+		
+	}
+}
+
+void kill_init()
+{
+}
+
+void kill_destroy()
+{
+	if( cut_buffer )
+		free( cut_buffer );
+	
+	if( kill_current != 0 )
+	{
+		kill_current = kill_last->prev;
+		kill_last->prev = 0;
+		
+		while( kill_current )
+		{
+			ll_node_t *tmp = kill_current;
+			kill_current = kill_current->prev;
+			free( tmp->data );
+			free( tmp );
+		}
+	}	
+	
+}
+

kill.h

@@ -0,0 +1,31 @@
+/** \file kill.h
+	Prototypes for the killring.
+
+	Works like the killring in emacs and readline. The killring is cut and paste whith a memory of previous cuts.
+*/
+
+/**
+   Add a string to the top of the killring
+*/
+void kill_add( wchar_t *str );
+/**
+   Rotate the killring
+*/
+wchar_t *kill_yank_rotate();
+/**
+  Paste from the killring
+*/
+wchar_t *kill_yank();
+/**
+   Sanity check
+*/
+void kill_sanity_check();
+/**
+   Initialize the killring
+*/
+void kill_init();
+/**
+   Destroy the killring
+*/
+void kill_destroy();
+

main.c

@@ -0,0 +1,306 @@
+/*
+Copyright (C) 2005 Axel Liljencrantz
+
+This program is free software; you can redistribute it and/or
+modify it under the terms of the GNU General Public License
+as published by the Free Software Foundation; either version 2
+of the License, or (at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+*/
+
+
+/** \file main.c
+	The main loop of <tt>fish</tt>.
+*/
+
+#include "config.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <wchar.h>
+#include <string.h>
+#include <unistd.h>
+#include <errno.h>
+#include <unistd.h>
+#include <termios.h>
+#include <sys/types.h>
+#include <fcntl.h>
+
+#ifdef HAVE_GETOPT_H
+#include <getopt.h>
+#endif
+
+#include <locale.h>
+#include <signal.h>
+
+#include "util.h"
+#include "common.h"
+#include "reader.h"
+#include "builtin.h"
+#include "function.h"
+#include "complete.h"
+#include "wutil.h"
+#include "env.h"
+#include "sanity.h"
+#include "proc.h"
+#include "parser.h"
+#include "expand.h"
+#include "intern.h"
+
+/**
+   Parse init files
+*/
+static int read_init()
+{
+	char cwd[4096];
+	wchar_t *wcwd;
+	
+	if( !getcwd( cwd, 4096 ) )
+	{
+		wperror( L"getcwd" );		
+		return 0;
+	}
+
+	env_set( L"__fish_help_dir", DOCDIR, 0);	
+ 	
+	eval( L"builtin cd /etc 2>/dev/null; . fish 2>/dev/null", 0, TOP );
+	eval( L"builtin cd 2>/dev/null;. .fish 2>/dev/null", 0, TOP );
+	
+	if( chdir( cwd ) == -1 )
+	{
+//		fwprintf( stderr, L"Invalid directory: %s\n", cwd );
+//		wperror( L"chdir" );
+//		return 0;
+	}	
+	wcwd = str2wcs( cwd );
+	if( wcwd )
+	{
+		env_set( L"PWD", wcwd, ENV_EXPORT );
+		free( wcwd );
+	}
+
+	return 1;
+}
+
+/**
+   Calls a bunch of init functions, parses the init files and then
+   parses commands from stdin or files, depending on arguments
+*/
+
+int main( int argc, char **argv )
+{
+	int res=1;
+	int force_interactive=0;
+	int my_optind;
+	
+	char *cmd=0;	
+
+	fish_setlocale( LC_ALL, L"" );
+	is_interactive_session=1;
+	program_name=L"fish";
+	
+	while( 1 )
+	{
+#ifdef __GLIBC__
+		static struct option
+			long_options[] =
+			{
+				{
+					"command", required_argument, 0, 'c' 
+				}
+				,
+				{
+					"interactive", no_argument, 0, 'i' 
+				}
+				,
+				{
+					"profile", required_argument, 0, 'p' 
+				}
+				,
+				{
+					"help", no_argument, 0, 'h' 
+				}
+				,
+				{
+					"version", no_argument, 0, 'v' 
+				}
+				,
+				{ 
+					0, 0, 0, 0 
+				}
+			}
+		;
+		
+		int opt_index = 0;
+		
+		int opt = getopt_long( argc,
+							   argv, 
+							   "hivc:p:", 
+							   long_options, 
+							   &opt_index );
+		
+#else	
+		int opt = getopt( argc,
+						  argv, 
+						  "hivc:p:" );
+#endif
+		if( opt == -1 )
+			break;
+		
+		switch( opt )
+		{
+			case 0:
+				break;
+				
+			case 'c':		
+				cmd = optarg;				
+				is_interactive_session = 0;
+				break;
+
+			case 'h':
+				cmd = "help";
+				//interactive=0;
+				
+				break;
+
+			case 'i':
+				force_interactive = 1;
+				break;				
+				
+			case 'p':
+				profile = optarg;
+				break;				
+				
+			case 'v':
+				fwprintf( stderr, 
+						  L"%s, version %s\n", 
+						  PACKAGE_NAME,
+						  PACKAGE_VERSION );
+				exit( 0 );				
+
+			case '?':
+				return 1;
+				
+		}		
+	}
+
+	my_optind = optind;
+	
+	is_login |= strcmp( argv[0], "-fish") == 0;
+//	fwprintf( stderr, L"%s\n", argv[0] );
+		
+	is_interactive_session &= (cmd == 0);
+	is_interactive_session &= (my_optind == argc);
+	is_interactive_session &= isatty(STDIN_FILENO);	
+
+//	fwprintf( stderr, L"%d %d %d\n", cmd==0, my_optind == argc, isatty(STDIN_FILENO) );
+
+	if( force_interactive )
+		is_interactive_session=1;	
+	
+	parser_init();
+	builtin_init();
+	function_init();
+	env_init();
+	complete_init();
+	reader_init();	
+
+	reader_push_current_filename( L"(internal)" );
+
+	if( read_init() )
+	{
+		if( cmd != 0 )
+		{
+			wchar_t *cmd_wcs = str2wcs( cmd );
+			res = eval( cmd_wcs, 0, TOP );
+			free(cmd_wcs);
+			reader_exit(0);			
+		}
+		else
+		{
+			if( my_optind == argc )
+			{
+				reader_push_current_filename( L"(stdin)" );
+				res = reader_read();				
+				reader_pop_current_filename();
+			}
+			else
+			{
+				char **ptr; 
+				char *file = *(argv+1);
+				int i; 
+				string_buffer_t sb;
+
+				if( close( 0 ) )
+				{
+					wperror(L"close");
+					return 1;
+				}
+				if( open(file, O_RDONLY) == -1 )
+				{
+					wperror( L"open" );
+					return 1;
+				}
+
+				sb_init( &sb );
+				
+				if( *(argv+2))
+				{
+					for( i=1,ptr = argv+2; *ptr; i++, ptr++ )
+					{
+						if( i != 1 )
+							sb_append( &sb, ARRAY_SEP_STR );
+						wchar_t *val = str2wcs( *ptr );
+						sb_append( &sb, val );
+						free( val );
+					}
+				
+					env_set( L"argv", (wchar_t *)sb.buff, 0 );
+					sb_destroy( &sb );
+				}
+				
+				reader_push_current_filename( str2wcs( file ) );
+				res = reader_read();
+
+				if( res )
+				{
+					debug( 1, 
+						   L"Error while reading file %ls\n", 
+						   reader_current_filename() );
+				}				
+				free(reader_pop_current_filename());
+			}
+		}
+	}
+
+	
+	if( function_exists(L"fish_on_exit"))
+	{
+		eval( L"fish_on_exit", 0, TOP );
+	}
+	job_do_notification();		
+	
+	reader_pop_current_filename();	
+	
+	proc_destroy();
+	env_destroy();
+	builtin_destroy();
+	function_destroy();
+	complete_destroy();
+	reader_destroy();
+	parser_destroy();
+	wutil_destroy();
+	common_destroy();
+	
+	intern_free_all();
+
+	return res;	
+}

output.c

@@ -0,0 +1,367 @@
+#include "config.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <termios.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/ioctl.h>
+#include <sys/time.h>
+#include <unistd.h>
+#include <wctype.h>
+
+#if HAVE_NCURSES_H
+#include <ncurses.h>
+#else
+#include <curses.h>
+#endif
+
+#if HAVE_TERMIO_H
+#include <termio.h>
+#endif
+
+#include <term.h>
+#include <signal.h>
+#include <fcntl.h>
+#include <dirent.h>
+#include <time.h>
+#include <wchar.h>
+
+#include "util.h"
+#include "wutil.h"
+#include "expand.h"
+#include "common.h"
+#include "output.h"
+#include "highlight.h"
+
+/**
+   Number of color names in the col array
+*/
+#define COLORS (sizeof(col)/sizeof(wchar_t *))
+
+/**
+   Names of different colors. 
+*/
+static wchar_t *col[]=
+{
+	L"black",
+	L"red",
+	L"green",
+	L"brown",
+	L"yellow",
+	L"blue",
+	L"magenta",
+	L"purple",
+	L"cyan",
+	L"white"
+	L"normal"
+}
+	;
+
+/**
+   Mapping from color name (the 'col' array) to color index as used in
+   ANSI color terminals, and also the fish_color_* constants defined
+   in highlight.h. Non-ANSI terminals will display the wrong colors,
+   since they use a different mapping.
+*/
+static int col_idx[]=
+{
+	0, 
+	1,
+	2,
+	3,
+	3,
+	4,
+	5,
+	5,
+	6,
+	7,
+	FISH_COLOR_NORMAL,
+}
+	;
+
+void set_color( int c, int c2 )
+{
+	static int last_color = FISH_COLOR_NORMAL, last_color2=FISH_COLOR_NORMAL;
+	int bg_set=0, last_bg_set=0;
+	char *fg = 0, *bg=0;
+
+	if( (set_a_foreground != 0) && (strlen( set_a_foreground) != 0 ) )
+	{
+		fg = set_a_foreground;
+		bg = set_a_background;
+	}
+	else if( (set_foreground != 0) && (strlen( set_foreground) != 0 ) )
+	{
+		fg = set_foreground;
+		bg = set_background;
+	}
+
+	if( (c == FISH_COLOR_RESET) || (c2 == FISH_COLOR_RESET))
+	{
+		c = c2 = FISH_COLOR_NORMAL;
+		if( fg )
+			writembs( tparm( set_a_foreground, 0 ) );
+		writembs( exit_attribute_mode );
+		return;
+	}
+
+	if( last_color2 != FISH_COLOR_NORMAL &&
+		last_color2 != FISH_COLOR_RESET &&
+		last_color2 != FISH_COLOR_IGNORE )
+	{
+		/*
+		  Background was set
+		*/
+		last_bg_set=1;
+	}
+
+	if( c2 != FISH_COLOR_NORMAL &&
+		c2 != FISH_COLOR_RESET &&
+		c2 != FISH_COLOR_IGNORE )
+	{
+		/*
+		  Background is set
+		*/
+		bg_set=1;
+		c = (c2==FISH_COLOR_WHITE)?FISH_COLOR_BLACK:FISH_COLOR_WHITE;
+	}
+
+	if( (enter_bold_mode != 0) && (strlen(enter_bold_mode) > 0))
+	{
+		if(bg_set && !last_bg_set)
+		{
+			/*
+			  Background color changed and is set, so we enter bold mode to make reading easier
+			*/
+			writembs( enter_bold_mode );
+		}
+		if(!bg_set && last_bg_set)
+		{
+			/*
+			  Background color changed and is no longer set, so we exit bold mode
+			*/
+			writembs( exit_attribute_mode );
+			/*
+			  We don't know if exit_attribute_mode resets colors, so
+			  we set it to something known.
+			*/
+			if( fg )
+			{
+				writembs( tparm( fg, 0 ) );
+				last_color=0;
+			}
+		}
+	}
+
+	if( last_color != c )
+	{
+		if( c==FISH_COLOR_NORMAL )
+		{
+			if( fg )
+				writembs( tparm( fg, 0 ) );
+			writembs( exit_attribute_mode );
+
+			last_color2 = FISH_COLOR_NORMAL;
+		}
+		else if( ( c >= 0) && ( c < FISH_COLOR_NORMAL ) )
+		{
+			if( fg )
+			{
+				writembs( tparm( fg, c ) );
+			}
+		}
+	}
+
+	last_color = c;
+
+	if( last_color2 != c2 )
+	{
+		if( c2 == FISH_COLOR_NORMAL )
+		{
+			if( bg )
+			{
+				writembs( tparm( bg, 0 ) );
+			}
+
+			writembs(exit_attribute_mode);
+			if(( last_color != FISH_COLOR_NORMAL ) && fg )
+			{
+				writembs(tparm( fg, last_color ));
+			}
+
+			last_color2 = c2;
+		}
+		else if ((c2 >= 0 ) &&(c2 < FISH_COLOR_NORMAL))
+		{
+			if( bg )
+			{
+				writembs( tparm( bg, c2 ) );
+			}
+			last_color2 = c2;
+		}
+	}
+}
+
+int writembs( char *str )
+{
+#ifdef TPUTS_KLUDGE
+	write( 1, str, strlen(str));
+#else
+	tputs(str,1,&writeb);
+#endif
+	return 0;
+}
+
+/**
+   Write a wide character to fd 1.
+*/
+int writech( wint_t ch )
+{
+	static mbstate_t out_state;
+	char buff[MB_CUR_MAX];
+	size_t bytes = wcrtomb( buff, ch, &out_state );
+	int err;
+	
+	while( (err =write( 1, buff, bytes ) ) )
+	{
+		if( err >= 0 )
+			break;
+		
+		if( errno == EINTR )
+			continue;
+		
+		wperror( L"write" );
+		return 1;
+	}
+	
+	return 0;
+}
+
+/**
+   Write a wide character string to FD 1.
+*/
+void writestr( const wchar_t *str )
+{
+	while( *str != 0 )
+		writech( *(str++) );
+}
+
+
+/**
+   Write a wide character string to FD 1. If the string is wider than
+   the specified maximum, truncate and ellipsize it.
+*/
+void writestr_ellipsis( const wchar_t *str, int max_width )
+{
+	int written=0;
+	int tot = my_wcswidth(str);
+
+	if( tot <= max_width )
+	{
+		writestr( str );
+		return;
+	}
+
+	while( *str != 0 )
+	{
+		int w = wcwidth( *str );
+		if( written+w+wcwidth( ellipsis_char )>max_width )
+			break;
+		written+=w;
+		writech( *(str++) );
+	}
+
+	written += wcwidth( ellipsis_char );
+	writech( ellipsis_char );
+
+	while( written < max_width )
+	{
+		written++;
+		writestr( L" " );
+	}
+}
+
+/**
+   Escape and write a string to fd 1
+*/
+int write_escaped_str( const wchar_t *str, int max_len )
+{
+
+	wchar_t *out = escape( wcsdup(str), 1 );
+	int i;
+	int len = my_wcswidth( out );
+	int written=0;
+
+	if( max_len && (max_len < len))
+	{
+		for( i=0; (written+wcwidth(out[i]))<=(max_len-1); i++ )
+		{
+			writech( out[i] );
+			written += wcwidth( out[i] );
+		}
+		writech( ellipsis_char );
+		written += wcwidth( ellipsis_char );
+		
+		for( i=written; i<max_len; i++ )
+		{
+			writech( L' ' );
+			written++;
+		}
+	}
+	else
+	{
+		written = len;
+		writestr( out );
+	}
+
+	free( out );
+	return written;
+}
+
+
+/**
+   parm_ich seems to often be undefined, so we use this
+   workalike. Writes the specified number of spaces.
+*/
+int writespace( int c )
+{
+	if( repeat_char && strlen(repeat_char) )
+	{
+		debug( 1, L"YAY" );
+		
+		writembs( tparm( repeat_char, ' ', c ) );
+	}
+	else
+	{
+		write( 1, "        ", mini(c,8) );
+		if( c>8)
+		{
+			writespace( c-8);
+		}
+	}
+	
+	return 0;
+}
+
+
+int output_color_code( const wchar_t *val )
+{
+	int i, color=-1;
+	
+	for( i=0; i<COLORS; i++ )
+	{
+		if( wcscasecmp( col[i], val ) == 0 )
+		{
+			color = col_idx[i];
+			break;
+		}
+	}
+
+	if( color >= 0 )
+		return color;
+	else
+		return FISH_COLOR_NORMAL;
+}

output.h

@@ -0,0 +1,89 @@
+/**
+   Constants for various character classifications. Each character of a command string can be classified as one of the following types.
+*/
+enum
+{
+	HIGHLIGHT_NORMAL,
+	HIGHLIGHT_COMMAND,
+	HIGHLIGHT_SUBSHELL,
+	HIGHLIGHT_REDIRECTION, 
+	HIGHLIGHT_END, 
+	HIGHLIGHT_ERROR,
+	HIGHLIGHT_PARAM,
+	HIGHLIGHT_COMMENT,
+	HIGHLIGHT_MATCH,
+	HIGHLIGHT_SEARCH_MATCH,
+}
+	;
+
+/**
+   Constants for various colors as used by the set_color function. 
+*/
+enum
+{
+	FISH_COLOR_BLACK,
+	FISH_COLOR_RED,
+	FISH_COLOR_GREEN,
+	FISH_COLOR_YELLOW,
+	FISH_COLOR_BLUE,
+	FISH_COLOR_MAGENTA,
+	FISH_COLOR_CYAN,
+	FISH_COLOR_WHITE,
+	/** The default fg color of the terminal */
+	FISH_COLOR_NORMAL
+}
+;
+ 
+
+/**
+   Sets the fg and bg color. May be called as often as you like, since
+   if the new color is the same as the previous, nothing will be
+   written. Negative values for set_color will also be ignored. Since
+   the terminfo string this function emits can potentially cause the
+   screen to flicker, the function takes care to write as little as
+   possible.
+
+   Possible values for color are any form the FISH_COLOR_* enum,
+   FISH_COLOR_IGNORE and FISH_COLOR_RESET. FISH_COLOR_IGNORE will
+   leave the color unchanged, and FISH_COLOR_RESET will perform an
+   exit_attribute_mode, even if set_color thinks it is already in
+   FISH_COLOR_NORMAL mode.
+
+   In order to set the color to normal, three terminfo strings may
+   have to be written.
+
+   - First a string to set the color, such as set_a_foreground. This
+   is needed because otherwise the previous strings colors might be
+   removed as well.
+
+   - After that we write the exit_attribute_mode string to reset all
+   color attributes.
+
+   - Lastly we may need to write set_a_background or set_a_foreground
+   to set the other half of the color pair to what it should be.
+
+   \param c Foreground color.
+   \param c2 Background color.
+*/
+
+
+void set_color( int c, int c2 );
+
+/**
+   Write a char * narrow string to FD 1, needed for the terminfo
+   strings.
+*/
+int writembs( char *str );
+
+int writech( wint_t ch );
+
+void writestr( const wchar_t *str );
+
+void writestr_ellipsis( const wchar_t *str, int max_width );
+
+int write_escaped_str( const wchar_t *str, int max_len );
+
+int writespace( int c );
+
+int output_color_code( const wchar_t *val );
+

parser.h

@@ -0,0 +1,261 @@
+/** \file parser.h
+	The fish parser. 	
+*/
+
+/**
+   block_t represents a block of commands. 
+*/
+typedef struct block
+{
+	int type; /**< Type of block. Can be one of WHILE, FOR, IF and FUNCTION */
+	int skip; /**< Whether execution of the commands in this block should be skipped */
+	int tok_pos; /**< The start index of the block */
+
+	/**
+	   Status for the current loop block. Can be anu of the values from the loop_status enum.
+	*/
+	int loop_status;
+
+	/**
+	   First block type specific variable
+	*/
+	union 
+	{
+		int while_state;  /**< True if the loop condition has not yet been evaluated*/
+		wchar_t *for_variable; /**< Name of the variable to loop over */
+		int if_state; /**< The state of the if block */
+		wchar_t *switch_value; /**< The value to test in a switch block */
+		wchar_t *function_name; /**< The name of the function to define */
+	};	
+
+	/**
+	   Second block type specific variable
+	*/
+	union
+	{
+		array_list_t for_vars; /**< List of values for a for block */	
+		int switch_taken; /**< Whether a switch match has already been found */
+		wchar_t *function_description; /**< The description of the function to define */
+	};
+
+    /**
+	   Next outer block 
+	*/
+	struct block *outer; 
+} block_t;
+
+/** 
+	Types of blocks
+*/
+enum block_type
+{
+	WHILE, /**< While loop block */
+	FOR,  /**< For loop block */
+	IF, /**< If block */
+	FUNCTION_DEF, /**< Function definition block */
+	FUNCTION_CALL, /**< Function invocation block */
+	SWITCH, /**< Switch block */
+	FAKE, /**< Fake block */
+	SUBST, /**< Command substitution scope */
+	TOP, /**< Outermost block */
+	BEGIN, /**< Unconditional block */
+	AND, /**< And block */
+	OR, /**< Or block */
+}
+;
+
+/**
+   Possible states for a loop
+*/
+enum loop_status 
+{
+	LOOP_NORMAL, /**< Current loop block executed as normal */
+	LOOP_BREAK, /**< Current loop block should be removed */
+	LOOP_CONTINUE, /**< Current loop block should be skipped */
+};
+
+
+/**
+   Possible states for a while block
+*/
+enum while_status
+{
+	WHILE_TEST_FIRST, /**< This is the first command of the first lap of a while loop */
+	WHILE_TEST_AGAIN, /**< This is not the first lap of the while loop, but it is the first command of the loop */
+	WHILE_TESTED, /**< This is not the first command in the loop */
+}
+;
+
+
+
+/**
+   Errors that can be generated by the parser
+*/
+enum parser_error 
+{
+	NO_ERR=0,
+	SYNTAX_ERROR,
+	EVAL_ERROR,
+	OOM,
+	STACK_ERROR,
+	SUBSHELL_ERROR
+}
+;
+
+/** The current innermost block */
+extern block_t *current_block;
+
+/** The current error code */
+extern int error_code;
+
+/**
+   Current block level redirections 
+*/
+extern io_data_t *block_io;
+
+/**
+  Finds the full path of an executable in a newly allocated string.
+  
+  \param cmd The name of the executable.
+  \return 0 if the command can not be found, the path of the command otherwise.
+*/
+wchar_t *get_filename( const wchar_t *cmd );
+
+/**
+  Evaluate the expressions contained in cmd.
+
+  \param cmd the string to evaluate
+  \param out buffer to insert output to. May be null.
+  \param the type of block to push onto the scope stack
+  \param block_type The type of block to push on the block stack
+  \return 0 on success.
+*/
+int eval( const wchar_t *cmd, io_data_t *io, int block_type );
+
+/**
+  Evaluate line as a list of parameters, i.e. tokenize it and perform parameter expansion and subshell execution on the tokens.
+  The output is inserted into output, and should be freed by the caller.
+
+  \param line Line to evaluate
+  \param output List to insert output to
+*/
+int eval_args( const wchar_t *line,
+				array_list_t *output );
+
+/**
+   Sets the current error
+
+   \param ec The new error code
+   \param str The new error message
+   \param p The character offset at which the error occured
+*/
+void error( int ec, const wchar_t *str, int p );
+
+/**
+   Tests if the specified commands parameters should be interpreted as another command, which will be true if the command is either 'command', 'exec', 'if', 'while' or 'builtin'.  
+
+   \param cmd The command name to test
+   \return 1 of the command parameter is a command, 0 otherwise
+*/
+
+int parser_is_subcommand( const wchar_t *cmd );
+
+/**
+   Tests if the specified command is a reserved word, i.e. if it is
+   the name of one of the builtin functions that change the block or
+   command scope, like 'for', 'end' or 'command' or 'exec'. These
+   functions may not be overloaded, so their names are reserved.
+
+   \param cmd The command name to test
+   \return 1 of the command parameter is a command, 0 otherwise
+*/
+int parser_is_reserved( wchar_t *word);
+
+/**
+   Returns a string describing the current parser pisition in the format 'FILENAME (line LINE_NUMBER): LINE'.
+   Example: 
+
+   init.fish (line 127): ls|grep pancake
+*/
+wchar_t *parser_current_line();
+
+/**
+   Returns the current position in the latest string of the tokenizer.
+*/
+int parser_get_pos();
+
+/**
+   Returns the position where the current job started in the latest string of the tokenizer.
+*/
+int parser_get_job_pos();
+
+/**
+   Set the current position in the latest string of the tokenizer.
+*/
+void parser_set_pos( int p);
+
+/**
+   Get the string currently parsed
+*/
+const wchar_t *parser_get_buffer();
+
+/**
+   Create block of specified type
+*/
+void parser_push_block( int type);
+
+/**
+   Remove the outermost block namespace
+*/
+void parser_pop_block();
+
+/**
+   Return a description of the given blocktype
+*/
+wchar_t *parser_get_block_desc( int block );
+
+
+/**
+   Test if the specified string can be parsed, or if more bytes need to be read first. 
+   The result has the first bit set if the string contains errors, and the second bit is set if the string contains an unclosed block.
+*/
+int parser_test( wchar_t * buff, int babble );
+
+/**
+   Returns the full path of the specified directory. If the \c in is a
+   full path to an existing directory, a copy of the string is
+   returned. If \c in is a directory relative to one of the
+   directories i the CDPATH, the full path is returned. If no
+   directory can be found, 0 is returned.
+*/
+wchar_t *parser_cdpath_get( wchar_t *in );
+
+/**
+   Tell the parser that the specified function may not be run if not
+   inside of a conditional block. This is to remove some possibilities
+   of infinite recursion.
+*/
+void parser_forbid_function( wchar_t *function );
+/**
+   Undo last call to parser_forbid_function().
+*/
+void parser_allow_function();
+
+/**
+   Initialize the parser
+*/
+void parser_init();
+
+/**
+   Destroy the parser
+*/
+void parser_destroy();
+
+/**
+   This function checks if the specified string is a help option. 
+
+   \param s the string to test
+   \param min_match is the minimum number of characters that must match in a long style option, i.e. the longest common prefix between --help and any other option. If less than 3, 3 will be assumed.
+*/
+int parser_is_help( wchar_t *s, int min_match );
+

proc.h

@@ -0,0 +1,264 @@
+/** \file proc.h 
+
+    Prototypes for utilities for keeping track of jobs, processes and subshells, as
+	well as signal handling functions for tracking children. These
+	functions do not themselves launch new processes, the exec library
+	will call proc to create representations of the running jobs as
+	needed.
+	
+*/
+
+/**
+   Describes what type of IO operation an io_data_t represents
+*/
+enum io_mode
+{
+	IO_FILE, IO_PIPE, IO_FD, IO_BUFFER, IO_CLOSE
+}
+;
+
+/**
+   Types of internal processes
+*/
+enum
+{
+	EXTERNAL,
+	INTERNAL_BUILTIN,
+	INTERNAL_FUNCTION,
+	INTERNAL_BLOCK,
+	INTERNAL_EXEC
+}
+	;
+
+
+/** Represents an FD redirection */
+typedef struct io_data
+{
+	/** Type of redirect */
+	int io_mode;
+	/** FD to redirect */
+	int fd;
+	/** parameter for redirection */
+	union
+	{
+		/** Fds for IO_PIPE and for IO_BUFFER */
+		int pipe_fd[2];
+		/** Filename IO_FILE */
+		wchar_t *filename;
+		/** fd to redirect specified fd to, for IO_FD*/
+		int old_fd;
+	}
+	;
+	union
+	{
+		/** file creation flags to send to open for IO_FILE */
+		int flags;
+		/** buffer to save output in for IO_BUFFER */
+		buffer_t *out_buffer;		
+		/** Whether to close old_fd for IO_FD */
+		int close_old;
+		
+	}
+	;
+	
+	/** Pointer to the next IO redirection */
+	struct io_data *next;
+}
+io_data_t;
+
+ 
+/** 
+	A structore representing a single process. Contains variables for
+	tracking process state and the process argument list. 
+*/
+typedef struct process{
+	/** argv parameter for for execv */
+	wchar_t **argv;
+	/** actual command to pass to exec */
+	wchar_t *actual_cmd;       
+	/** process ID */
+	pid_t pid;
+	/** 
+		Type of process. Can be one of \c EXTERNAL, \c
+		INTERNAL_BUILTIN, \c INTERNAL_FUNCTION, \c INTERNAL_BLOCK
+	*/
+	int type;
+	/** true if process has completed */
+	volatile int completed;
+	/** true if process has stopped */
+	volatile int stopped;
+	/** reported status value */
+	volatile int status;
+	/** next process in pipeline */
+	struct process *next;       	
+#ifdef HAVE__PROC_SELF_STAT
+	/** Last time of cpu time check */
+	struct timeval last_time;
+	/** Number of jiffies spent in process at last cpu time check */
+	unsigned long last_jiffies;	
+#endif
+} process_t;
+
+
+/** Represents a pipeline of one or more processes.  */
+typedef struct job
+{
+	/** command line, used for messages */
+	wchar_t *command;              
+	/** list of processes in this job */
+	process_t *first_process;  
+	/** process group ID */
+	pid_t pgid;   
+	/** true if user was told about stopped job */
+	int notified;     
+	/** saved terminal modes */
+	struct termios tmodes;    
+	/** The job id of the job*/
+	int job_id;
+	/** Whether this job is in the foreground */
+	int fg;
+	/** 
+		Whether the specified job is completely constructed,
+		i.e. completely parsed, and every process in the job has been
+		forked
+	*/
+	int constructed;
+	/**
+	   Whether the specified job is a part of a subshell or some other form of special job that should not be reported
+	*/
+	int skip_notification;
+	
+	/** List of IO redrections for the job */
+	io_data_t *io;
+	
+	/** Should the exit status be negated */
+	int negate;	
+	/** Is this a conditional short circut thing? If so, is it an COND_OR or a COND_AND */
+	struct job *next;           
+} job_t;
+
+/** Whether we are running a subshell command */
+extern int is_subshell;
+/** Whether we are running a block of commands */
+extern int is_block;
+/** Whether we are reading from the keyboard right now*/
+extern int is_interactive;
+/** Whether this shell is attached to the keyboard at all*/
+extern int is_interactive_session;
+/** Whether we are a login shell*/
+extern int is_login;
+/** Linked list of all jobs */
+extern job_t *first_job;   
+
+
+extern pid_t proc_last_bg_pid;
+
+/**
+   Join two chains of io redirections
+*/
+io_data_t *io_add( io_data_t *first_chain, io_data_t *decond_chain );
+
+/**
+   Remove the specified io redirection from the chain
+*/
+io_data_t *io_remove( io_data_t *list, io_data_t *element );
+
+/**
+   Make a copy of the specified chain of redirections
+*/
+io_data_t *io_duplicate( io_data_t *l );
+
+/**
+   Return the last io redirection in ht e chain for the specified file descriptor.
+*/
+io_data_t *io_get( io_data_t *io, int fd );
+
+/**
+   Sets the status of the last process to exit
+*/
+void proc_set_last_status( int s );
+/**
+   Returns the status of the last process to exit
+*/
+int proc_get_last_status();
+
+/**
+   Remove the specified job
+*/
+void job_free( job_t* j );
+/**
+   Create a new job
+*/
+job_t *job_create();
+
+/**
+  Return the job with the specified job id.
+  If id is -1, return the last job used.
+*/
+job_t *job_get(int id);
+
+/**
+  Return the job with the specified pid.
+*/
+job_t *job_get_from_pid(int pid);
+
+/**
+   Tests if the job is stopped 
+ */
+int job_is_stopped( const job_t *j );
+
+/**
+   Tests if the job has completed
+ */
+int job_is_completed( const job_t *j );
+
+/**
+  Reassume a (possibly) stopped job. Put job j in the foreground.  If
+  cont is nonzero, restore the saved terminal modes and send the
+  process group a SIGCONT signal to wake it up before we block.
+
+  \param j The job
+  \param cont Whether the function should wait for the job to complete before returning
+*/
+void job_continue( job_t *j, int cont );
+/**
+   Notify user of nog events.  Notify the user about stopped or
+   terminated jobs.  Delete terminated jobs from the active job list.
+*/
+int job_do_notification();
+/**
+   Signal handler for SIGCHLD.  Mark any processes with relevant
+   information.
+
+*/
+void job_handle_signal( int signal, siginfo_t *info, void *con );
+
+/**
+   Clean up before exiting
+*/
+void proc_destroy();
+
+
+#ifdef HAVE__PROC_SELF_STAT
+/**
+   Use the procfs filesystem to look up how many jiffies of cpu time
+   was used by this process. This function is only available on
+   systems with the procfs file entry 'stat', i.e. Linux.
+*/
+unsigned long proc_get_jiffies( process_t *p );
+
+/**
+   Update process time usage for all processes by calling the
+   proc_get_jiffies function for every process of every job.
+*/
+void proc_update_jiffies();
+
+#endif
+
+/**
+   Perform a set of simple sanity checks on the job list. This
+   includes making sure that only one job is in the foreground, that
+   every process is in a valid state, etc.
+*/
+void proc_sanity_check();
+

reader.h

@@ -0,0 +1,186 @@
+/** \file reader.h 
+
+    Prototypes for functions for reading data from stdin and passing
+	to the parser. If stdin is a keyboard, it supplies a killring,
+	history, syntax highlighting, tab-completion and various other
+	features.
+*/
+
+/**
+  Read commands from fd 0 until encountering EOF
+*/
+int reader_read();
+
+/**
+  Tell the shell that it should exit after the currently running command finishes.
+*/
+void reader_exit( int do_exit );
+
+/**
+   Check that the reader is in a sane state
+*/
+void reader_sanity_check();
+
+/**
+   Initialize the reader
+*/
+void reader_init();
+
+/**
+   Destroy and free resources used by the reader
+*/
+void reader_destroy();
+
+/**
+   Returns the filename of the file currently read
+*/
+wchar_t *reader_current_filename();
+
+/**
+   Push a new filename on the stack of read files
+   
+   \param fn The fileanme to push
+*/
+void reader_push_current_filename( wchar_t *fn );
+/**
+   Pop the current filename from the stack of read files
+ */
+wchar_t *reader_pop_current_filename();
+
+/** 
+	Returns the width of the terminal window, so that not all
+	functions that use these values continually have to keep track of
+	it.
+*/
+int reader_get_width();
+/**
+   Returns the height of the terminal window, so that not all
+   functions that use these values continually have to keep track of
+   it.
+*/
+int reader_get_height();
+
+/**
+   Write the title to the titlebar. This function is called just
+   before a new application starts executing and just after it
+   finishes.
+*/
+void reader_write_title();
+
+/**
+   Repaint the entire commandline. This means reset and clear the
+   commandline, write the prompt, perform syntax highlighting, write
+   the commandline and move the cursor.
+*/
+void repaint();
+
+/**
+   Run the specified command with the correct terminal modes, and
+   while taking care to perform job notification, set the title, etc.
+*/
+void reader_run_command( wchar_t *buff );
+
+/**
+   Get the string of character currently entered into the command
+   buffer, or 0 if interactive mode is uninitialized.
+*/
+wchar_t *reader_get_buffer();
+
+/**
+   Set the string of characters in the command buffer, as well as the cursor position.
+
+   \param b the new buffer value
+   \param p the cursor position. If \c p is less than zero, the cursor is placed on the last character.
+*/
+void reader_set_buffer( wchar_t *b, int p );
+
+/**
+   Get the current cursor position in the command line. If interactive
+   mode is uninitialized, return -1.
+*/
+int reader_get_cursor_pos();
+
+/**
+   Return the value of the interupted flag, which is set by the sigint
+   handler, and clear it if it was set.
+*/
+int reader_interupted();
+
+/**
+   Read one line of input. Before calling this function, reader_push()
+   must have been called in order to set up a valid reader
+   environment.
+*/
+wchar_t *reader_readline();
+
+/**
+   Push a new reader environment. 
+*/
+void reader_push( wchar_t *name );
+
+/**
+   Return to previous reader environment
+*/
+void reader_pop();
+
+/**
+   Specify function to use for finding possible tab completions. The function must take these arguments: 
+
+   - The command to be completed as a null terminated array of wchar_t
+   - An array_list_t in which completions will be inserted.
+*/
+void reader_set_complete_function( void (*f)( const wchar_t *, array_list_t * ) );
+
+/**
+   Specify function for syntax highlighting. The function must take these arguments:
+
+   - The command to be highlighted as a null terminated array of wchar_t
+   - The color code of each character as an array of ints
+   - The cursor position
+   - An array_list_t used for storing error messages
+*/
+void reader_set_highlight_function( void (*f)( wchar_t *, int *, int, array_list_t * ) );
+
+/**
+   Specify function for testing if the command buffer contains syntax
+   errors that must be corrected before returning.
+*/
+void reader_set_test_function( int (*f)( wchar_t * ) );
+
+/**
+   Specify string of shell commands to be run in order to generate the
+   prompt.
+*/
+void reader_set_prompt( wchar_t *prompt );
+
+/**
+   Returns true if the shell is exiting, 0 otherwise. 
+*/
+int exit_status();
+
+/**
+   Find the beginning and the end of the current subshell
+*/
+void reader_current_subshell_extent( wchar_t **a, wchar_t **b );
+
+/**
+   Find the beginning and the end of the job under the cursor 
+*/
+void reader_current_job_extent( wchar_t **a, wchar_t **b );
+
+/**
+   Find the beginning and the end of the process under the cursor 
+*/
+void reader_current_process_extent( wchar_t **a, wchar_t **b );
+
+/**
+   Find the beginning and the end of the token under the curor and the token before the cursor
+*/
+
+void reader_current_token_extent( wchar_t **a, wchar_t **b, wchar_t **pa, wchar_t **pb );
+
+/**
+   Replace the current token with the specified string
+*/
+void reader_replace_current_token( wchar_t *new_token );
+

sanity.c

@@ -0,0 +1,72 @@
+/** \file sanity.c
+	Functions for performing sanity checks on the program state
+*/
+#include <stdlib.h>
+#include <wchar.h>
+#include <stdio.h>
+#include <errno.h>
+#include <termios.h>
+#include <unistd.h>
+#include <signal.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <dirent.h>
+
+
+#include "config.h"
+#include "util.h"
+#include "common.h"
+#include "sanity.h"
+#include "proc.h"
+#include "history.h"
+#include "reader.h"
+#include "kill.h"
+#include "wutil.h"
+
+/**
+   Status from earlier sanity checks
+*/
+static int insane;
+
+void sanity_lose()
+{
+	debug( 0, L"Errors detected, shutting down" );
+	insane = 1;
+}
+
+int sanity_check()
+{
+	if( !insane )
+		if( is_interactive )
+			history_sanity_check();
+	if( !insane )
+		reader_sanity_check();
+	if( !insane )
+		kill_sanity_check();
+	if( !insane )
+		proc_sanity_check();
+	
+	return insane;
+}
+
+void validate_pointer( const void *ptr, const wchar_t *err, int null_ok )
+{
+	
+	/* 
+	   Test if the pointer data crosses a segment boundary. 
+	*/
+	
+	if( (0x00000003 & (int)ptr) != 0 )
+	{
+		debug( 0, L"The pointer '\%ls\' is invalid", err );
+		sanity_lose();		
+	}
+	
+	if((!null_ok) && (ptr==0))
+	{
+		debug( 0, L"The pointer '\%ls\' is null", err );
+		sanity_lose();		
+	}
+}
+
+

sanity.h

@@ -0,0 +1,22 @@
+/** \file sanity.h
+	Prototypes for functions for performing sanity checks on the program state
+*/
+
+/**
+  Call this function to tell the program it is not in a sane state.
+*/
+void sanity_lose();
+
+/**
+  Perform sanity checks, return 1 if program is in a sane state 0 otherwise.
+*/
+int sanity_check();
+
+/**
+  Try and determine if ptr is a valid pointer. If not, loose sanity.
+  
+  \param ptr The pointer to validate
+  \param err A description of what the pointer refers to, for use in error messages
+  \param null_ok Wheter the pointer is allowed to point to 0
+*/
+void validate_pointer( const void *ptr, const wchar_t *err, int null_ok );

set_color.c

@@ -0,0 +1,235 @@
+#include "config.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <termios.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+#if HAVE_NCURSES_H
+#include <ncurses.h>
+#else
+#include <curses.h>
+#endif
+
+
+#if HAVE_TERMIO_H
+#include <termio.h>
+#endif
+
+#include <term.h>
+#include <errno.h>
+
+#ifdef HAVE_GETOPT_H
+#include <getopt.h>
+#endif
+
+/*
+  Small utility for setting the color.
+  Usage: set_color COLOR
+  where COLOR is either an integer from 0 to seven or one of the strings in the col array.
+*/
+
+#define COLORS (sizeof(col)/sizeof(char *))
+
+char *col[]=
+{
+	"black",
+	"red",
+	"green",
+	"brown",
+	"yellow",
+	"blue",
+	"magenta",
+	"purple",
+	"cyan",
+	"white",
+	"normal"
+}
+;
+
+int col_idx[]=
+{
+	0, 
+	1,
+	2,
+	3,
+	3,
+	4,
+	5,
+	5,
+	6,
+	7,
+	8
+}
+;
+
+void print_help();
+
+int translate_color( char *str )
+{
+	char *endptr;
+	int color;
+
+	if( !str )
+		return -1;
+	
+
+	color = strtol( str, &endptr, 10 );
+	if(endptr<=str)
+	{
+		int i;
+		color = -1;
+		for( i=0; i<COLORS; i++ )
+		{
+			
+			if( strcasecmp( col[i], str ) == 0 )
+			{
+				color = col_idx[i];				
+				break;
+			}
+		}
+	}	
+	return color;
+	
+}
+
+int main( int argc, char **argv )
+{
+	char *bgcolor=0;
+	char *fgcolor=0;
+	int fg, bg;
+	
+	while( 1 )
+	{
+#ifdef __GLIBC__
+		static struct option
+			long_options[] =
+			{
+				{
+					"background", required_argument, 0, 'b' 
+				}
+				,
+				{
+					"help", no_argument, 0, 'h' 
+				}
+				,
+				{
+					"version", no_argument, 0, 'v' 
+				}
+				,
+				{ 
+					0, 0, 0, 0 
+				}
+			}
+		;		
+		
+		int opt_index = 0;
+		
+		int opt = getopt_long( argc,
+							   argv, 
+							   "b:hv", 
+							   long_options, 
+							   &opt_index );
+#else
+		int opt = getopt( argc,
+						  argv, 
+						  "b:hv" );
+#endif
+		if( opt == -1 )
+			break;
+			
+		switch( opt )
+		{
+			case 0:
+				break;
+				
+			case 'b':				
+				bgcolor = optarg;
+				break;
+			case 'h':
+				print_help();
+				exit(0);				
+								
+			case 'v':
+				fprintf( stderr, "set_color, version %s\n", PACKAGE_VERSION );
+				exit( 0 );								
+
+			case '?':
+				return 1;
+				
+		}
+		
+	}		
+	
+	switch( argc-optind)
+	{
+		case 0:
+//			printf( "no fg\n" );
+			break;
+			
+		case 1:
+			fgcolor=argv[optind];
+//			printf( "fg %s\n", fgcolor );
+			break;
+			
+		default:
+			printf( "set_color: Too many arguments\n" );
+			return 1;
+	}
+
+	if( !fgcolor && !bgcolor )
+	{
+		print_help();		
+		return 1;
+	}
+	
+		
+	fg = translate_color(fgcolor);
+	if( fgcolor && (fg==-1))
+	{
+		fprintf( stderr, "set_color: Unknown color %s\n", fgcolor );
+		return 1;		
+	}
+
+	bg = translate_color(bgcolor);
+	if( bgcolor && (bg==-1))
+	{
+		fprintf( stderr, "set_color: Unknown color %s\n", bgcolor );
+		return 1;		
+	}
+
+	setupterm( 0, STDOUT_FILENO, 0);
+	if( bgcolor )
+	{
+		if( bg == 8 )
+		{
+			putp( tparm( set_a_background, 0) );
+			putp( tparm(exit_attribute_mode) );		
+		}	
+	}
+
+	if( fgcolor )
+	{
+		if( fg == 8 )
+		{
+			putp( tparm( set_a_foreground, 0) );
+			putp( tparm(exit_attribute_mode) );		
+		}	
+		else
+		{
+			putp( tparm( set_a_foreground, fg) );
+		}
+	}
+	
+	if( bgcolor )
+	{
+		if( bg != 8 )
+		{
+			putp( tparm( set_a_background, bg) );
+		}
+	}
+	del_curterm( cur_term );
+}

tokenize.c

@@ -0,0 +1,138 @@
+/** file tokenize.c
+  Small utility command for tokenizing an argument.
+  \c tokenize is used for splitting a text string into separate parts (i.e. tokenizing) with a user supplied delimiter character. 
+*/
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <unistd.h>
+
+#include "config.h"
+
+#ifdef HAVE_GETOPT_H
+#include <getopt.h>
+#endif
+
+/**
+   Print help message
+*/
+void print_help();
+
+/**
+   Main program
+*/
+int main( int argc, char **argv )
+{
+	char *delim = " \t";
+	int empty_ok = 0;
+	int i;
+	extern int optind;	
+	
+	while( 1 )
+	{
+#ifdef __GLIBC__
+		static struct option
+			long_options[] =
+			{
+				{
+					"with-empty", no_argument, 0, 'e' 
+				}
+				,
+				{
+					"no-empty", no_argument, 0, 'n' 
+				}
+				,
+				{
+					"delimiter", required_argument, 0, 'd' 
+				}
+				,
+				{
+					"help", no_argument, 0, 'h' 
+				}
+				,
+				{
+					"version", no_argument, 0, 'v' 
+				}
+				,
+				{ 
+					0, 0, 0, 0 
+				}
+			}
+		;		
+		
+		int opt_index = 0;
+		
+		int opt = getopt_long( argc,
+							   argv, 
+							   "end:hv", 
+							   long_options, 
+							   &opt_index );
+#else
+		int opt = getopt( argc,
+						  argv, 
+						  "end:hv" );
+#endif
+		if( opt == -1 )
+			break;
+			
+		switch( opt )
+		{
+			case 0:
+				break;
+				
+			case 'e':				
+				empty_ok = 1;
+				break;
+
+			case 'n':				
+				empty_ok = 0;
+				break;
+
+			case 'd':				
+				delim = optarg;
+				break;
+			case 'h':
+				print_help();
+				exit(0);				
+								
+			case 'v':
+				printf( "tokenize, version %s\n", PACKAGE_VERSION );
+				exit( 0 );								
+
+			case '?':
+				return 1;
+				
+		}
+		
+	}		
+	
+	for( i=optind; i<argc; i++ )
+	{
+		char *curr;
+		int printed=0;
+		for( curr = argv[i]; *curr; curr++ )
+		{
+			if( strchr( delim, *curr )==0 )
+			{
+				printed = 1;
+				putchar( *curr );
+			}
+			else
+			{
+				if( empty_ok || printed )
+					putchar( '\n' );
+				printed=0;
+			}
+		}
+		if( printed )
+			putchar( '\n' );
+	}
+	
+}
+
+
+
+
+

tokenizer.c

@@ -0,0 +1,625 @@
+/** \file tokenizer.c
+
+    A specialized tokenizer for tokenizing the fish language. In the
+    future, the tokenizer should be extended to support marks,
+    tokenizing multiple strings and disposing of unused string
+    segments.
+*/
+
+#include "config.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <wchar.h>
+#include <wctype.h>
+#include <string.h>
+#include <sys/types.h>
+#include <dirent.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+#include "util.h"
+#include "wutil.h"
+#include "tokenizer.h"
+#include "common.h"
+#include "wildcard.h"
+
+/**
+   Error string for unexpected end of string
+*/
+#define EOL_ERROR L"Unexpected end of token"
+/**
+   Error string for mismatched parenthesis
+*/
+#define PARAN_ERROR L"Parenthesis mismatch"
+/**
+   Error string for invalid redirections
+*/
+#define REDIRECT_ERROR L"Invalid redirection"
+/**
+   Error string for invalid input
+*/
+#define INPUT_ERROR L"Invalid input"
+
+/**
+  Characters that separate tokens. They are ordered by frequency of occurrence to increase parsing speed.
+*/
+#define SEP L" \n;|#\t\r<>^&"
+/**
+   Tests if the tokenizer buffer is large enough to hold contents of
+   the specified length, and if not, reallocates the tokenizer buffer.
+
+   \return 0 if the system could not provide the memory needed, and 1 otherwise.
+*/
+const static wchar_t *tok_desc[] =
+{
+	L"Tokenizer not yet initialized",
+	L"Tokenizer error",
+	L"Invalid token",
+	L"String",
+	L"Pipe",
+	L"End of command",
+	L"Redirect output to file",
+	L"Append output to file",
+	L"Redirect input to file",
+	L"Redirect to file descriptor",
+	L"Run job in background",
+	L"Comment"
+}
+;
+
+/**
+   Make sure the tokenizer buffer have room for a token of the specified size.
+*/
+static int check_size( tokenizer *tok, size_t len )
+{
+	if( tok->last_len <= len )
+	{
+		wchar_t *tmp;
+		tok->last_len = len +1;
+		tmp = realloc( tok->last, sizeof(wchar_t)*tok->last_len );
+		if( tmp == 0 )
+		{
+			wperror( L"realloc" );
+			return 0;
+		}
+		tok->last = tmp;
+	}
+	return 1;
+}
+
+/**
+   Set the latest tokens string to be the specified error message
+*/
+static void tok_error( tokenizer *tok, const wchar_t *err )
+{
+	tok->last_type = TOK_ERROR;
+	if( !check_size( tok, wcslen( err)+1 ))
+	{
+		if( tok->last != 0 )
+			*tok->last=0;
+		return;
+	}
+
+	wcscpy( tok->last, err );
+}
+
+void tok_init( tokenizer *tok, const wchar_t *b, int flags )
+{
+//	fwprintf( stderr, L"CREATE: \'%ls\'\n", b );
+		
+
+	memset( tok, 0, sizeof( tokenizer) );
+	
+	tok ->last = 0;
+	tok ->last_len = 0;
+	tok->accept_unfinished = flags & TOK_ACCEPT_UNFINISHED;
+	tok->show_comments = flags & TOK_SHOW_COMMENTS;
+	tok->has_next=1;
+	
+
+	/* 
+	   Before we copy the buffer we need to check that it is not
+	   null. But before that, we need to init the tokenizer far enough
+	   so that errors can be properly flagged
+	*/
+	if( !b )
+	{
+		tok_error( tok, INPUT_ERROR );
+		return;
+	}
+
+	tok->has_next = (*b != L'\0');
+	tok->orig_buff = tok->buff = wcsdup(b);
+
+	if( !tok->orig_buff )
+	{
+		die_mem();
+		
+	}
+	
+	if( tok->accept_unfinished )
+	{
+		int l = wcslen( tok->orig_buff );
+		if( l != 0 )
+		{
+			if( tok->orig_buff[l-1] == L'\\' )
+				tok->orig_buff[l-1] = L'\0';
+		}
+	}
+
+	tok_next( tok );
+
+}
+
+void tok_destroy( tokenizer *tok )
+{
+	free( tok->last );
+	free( tok->orig_buff );
+}
+
+int tok_last_type( tokenizer *tok )
+{
+	return tok->last_type;
+}
+
+wchar_t *tok_last( tokenizer *tok )
+{
+	return tok->last;
+}
+
+int tok_has_next( tokenizer *tok )
+{
+/*	fwprintf( stderr, L"has_next is %ls \n", tok->has_next?L"true":L"false" );*/
+	return 	tok->has_next;
+}
+
+/**
+   Tests if this character can be a part of a string
+*/
+
+static int is_string_char( wchar_t c )
+{
+	if( !c ||  wcschr( SEP, c ) )
+	{
+		return 0;
+	}
+
+  return 1;
+}
+
+/**
+   Read the next token as a string
+*/
+static void read_string( tokenizer *tok )
+{
+	const wchar_t *start;
+	int len;
+	int mode=0;
+	wchar_t prev;
+	int do_loop=1;
+	int paran_count=0;
+
+	start = tok->buff;
+
+	while( 1 )
+	{
+		if( *tok->buff == L'\\' )
+		{
+			tok->buff++;
+			if( *tok->buff == L'\0' )
+			{
+				tok_error( tok, EOL_ERROR );
+				return;
+			}
+			tok->buff++;
+			continue;
+		}
+
+		/*
+		  The modes are as follows:
+
+		  0: regular text
+		  1: inside of subshell
+		  2: inside of array brackets
+		  3: inside of array brackets and subshell, like in '$foo[(ech'
+		*/
+		switch( mode )
+		{
+			case 0:
+			{
+				switch( *tok->buff )
+				{
+					case L'(':
+					{
+						paran_count=1;
+						mode = 1;
+						break;
+					}
+					
+					case L'[':
+					{
+						if( tok->buff != start )
+							mode=2;
+						break;
+					}
+
+					case L'\'':
+					case L'"':
+					{
+						
+						wchar_t *end = quote_end( tok->buff );
+						tok->last_quote = *tok->buff;
+						if( end )
+						{
+							tok->buff=end;
+						}
+						else
+						{
+							tok->buff += wcslen( tok->buff );
+							
+							if( (!tok->accept_unfinished) )
+							{
+								tok_error( tok, EOL_ERROR );
+								return;
+							}
+							do_loop = 0;
+							
+						}
+						break;
+					}
+
+					default:
+					{
+						if( !is_string_char(*(tok->buff)) )
+						{
+							do_loop=0;
+						}
+					}
+				}
+				break;
+			}
+		
+			case 3:
+			case 1:
+				switch( *tok->buff )
+				{
+					case L'\'':
+					case L'\"':
+					{
+						wchar_t *end = quote_end( tok->buff );
+						if( end )
+						{
+							tok->buff=end;
+						}
+						else
+							do_loop = 0;
+						break;
+					}
+
+					case L'(':
+						paran_count++;
+						break;
+					case L')':
+						paran_count--;
+						if( paran_count == 0 )
+						{
+							mode--;
+						}
+						break;
+					case L'\0':
+						do_loop = 0;
+						break;
+				}
+				break;
+			case 2:
+				switch( *tok->buff )
+				{
+					case L'(':
+						paran_count=1;
+						mode = 3;
+						break;
+
+					case L']':
+						mode=0;
+						break;
+
+					case L'\0':
+						do_loop = 0;
+						break;
+				}
+				break;
+		}
+
+		if( !do_loop )
+			break;
+
+		prev = *tok->buff;
+		tok->buff++;
+	}
+
+	if( (!tok->accept_unfinished) && (mode!=0) )
+	{
+		tok_error( tok, PARAN_ERROR );
+		return;
+	}
+
+
+	len = tok->buff - start;
+
+	if( !check_size( tok, len ))
+		return;
+
+	memcpy( tok->last, start, sizeof(wchar_t)*len );
+	tok->last[len] = L'\0';
+	tok->last_type = TOK_STRING;
+}
+
+/**
+   Read the next token as a comment.
+*/
+static void read_comment( tokenizer *tok )
+{
+	const wchar_t *start;
+	int len;
+
+	start = tok->buff;
+	while( *(tok->buff)!= L'\n' && *(tok->buff)!= L'\0' )
+		tok->buff++;
+
+	len = tok->buff - start;
+	if( !check_size( tok, len ))
+		return;
+
+	memcpy( tok->last, start, sizeof(wchar_t)*len );
+	tok->last[len] = L'\0';
+	tok->last_type = TOK_COMMENT;
+}
+
+/**
+   Read a FD redirect.
+*/
+static void read_redirect( tokenizer *tok, int fd )
+{
+	int mode = -1;
+	
+	if( (*tok->buff == L'>') ||
+		(*tok->buff == L'^') )
+	{
+		tok->buff++;
+		if( *tok->buff == *(tok->buff-1) )
+		{
+			tok->buff++;
+			mode = 1;
+		}
+		else
+		{
+			mode = 0;
+		}
+	}
+	else if( *tok->buff == L'<' )
+	{
+		tok->buff++;
+		mode = 2;
+	}
+	else
+	{
+		tok_error( tok, REDIRECT_ERROR);
+	}
+	
+	if( !check_size( tok, 2 ))
+	{
+		return;
+	}
+	
+	swprintf( tok->last, tok->last_len, L"%d", fd );
+	
+	if( *tok->buff == L'&' )
+	{
+		tok->buff++;
+		tok->last_type = TOK_REDIRECT_FD;
+	}
+	else
+	{
+		tok->last_type = TOK_REDIRECT_OUT + mode;
+	}
+}
+
+wchar_t tok_last_quote( tokenizer *tok )
+{
+	return tok->last_quote;
+}
+
+/**
+   Test if a character is whitespace. Differs from iswspace in that it
+   does not consider a newline to be whitespace.
+*/
+static int my_iswspace( wchar_t c )
+{
+	if( c == L'\n' )
+		return 0;
+	else
+		return iswspace( c );
+}
+
+
+const wchar_t *tok_get_desc( int type )
+{
+	return tok_desc[type];
+}
+
+
+void tok_next( tokenizer *tok )
+{
+//	fwprintf( stderr, L"tok_next on %ls (prev=%ls)\n", tok->orig_buff, tok_desc[tok->last_type] );
+
+	if( tok_last_type( tok ) == TOK_ERROR )
+	{
+		tok->has_next=0;	
+		return;	
+	}
+	
+	if( !tok->has_next )
+	{
+/*		wprintf( L"EOL\n" );*/
+		tok->last_type = TOK_END;
+		return;
+	}
+
+	while( my_iswspace(*(tok->buff) ) )
+		tok->buff++;
+
+	if( *tok->buff == L'#')
+	{
+		if( tok->show_comments )
+		{
+			tok->last_pos = tok->buff - tok->orig_buff;
+			read_comment( tok );
+			return;
+		}
+		else
+		{
+			while( *(tok->buff)!= L'\n' && *(tok->buff)!= L'\0' )
+				tok->buff++;
+		}
+
+		while( my_iswspace(*(tok->buff) ) )
+			tok->buff++;
+	}
+
+	tok->last_pos = tok->buff - tok->orig_buff;
+
+	switch( *tok->buff )
+	{
+
+		case L'|':
+			tok->last_type = TOK_PIPE;
+			tok->buff++;
+			break;
+		case L'\0':
+			tok->last_type = TOK_END;
+			/*fwprintf( stderr, L"End of string\n" );*/
+			tok->has_next = 0;
+			break;
+		case 13:
+		case L'\n':
+		case L';':
+			tok->last_type = TOK_END;
+			tok->buff++;
+			break;
+		case L'&':
+			tok->last_type = TOK_BACKGROUND;
+			tok->buff++;
+			break;
+
+		case L'>':
+			return read_redirect( tok, 1 );
+		case L'<':
+			return read_redirect( tok, 0 );
+		case L'^':
+			return read_redirect( tok, 2 );
+
+		default:
+			if( iswdigit( *tok->buff ) )
+			{
+				int fd = *tok->buff - L'0';
+				switch( *(tok->buff+1))
+				{
+					case L'>':
+					case L'<':
+						tok->buff++;
+						read_redirect( tok, fd );
+						return;
+						
+				}
+			}
+			read_string( tok );
+	}
+
+}
+
+wchar_t *tok_string( tokenizer *tok )
+{
+	return tok->orig_buff;
+}
+
+wchar_t *tok_first( const wchar_t *str )
+{
+	tokenizer t;
+	wchar_t *res=0;
+
+	tok_init( &t, str, 0 );
+	
+	switch( tok_last_type( &t ) )
+	{
+		case TOK_STRING:
+//				fwprintf( stderr, L"Got token %ls\n", tok_last( &t ));
+			res = wcsdup(tok_last( &t ));
+			break;
+		default:
+			break;
+	}
+
+	tok_destroy( &t );
+	return res;
+}
+
+
+int tok_get_pos( tokenizer *tok )
+{
+	return tok->last_pos;
+}
+
+
+void tok_set_pos( tokenizer *tok, int pos )
+{
+	tok->buff = tok->orig_buff + pos;
+	tok->has_next = 1;
+	tok_next( tok );
+}
+
+
+#ifdef TOKENIZER_TEST
+
+/**
+   This main function is used for compiling the tokenizer_test command, used for testing the tokenizer.
+*/
+int main( int argc, char **argv )
+{
+	tokenizer tok;
+	int i;
+	for ( i=1; i<argc; i++ )
+	{
+		wprintf( L"Tokenizing string %s\n", argv[i] );
+		for( tok_init( &tok, str2wcs(argv[i]), 0 ); tok_has_next( &tok ); tok_next( &tok ) )
+		{
+			switch( tok_last_type( &tok ) )
+ 			{
+				case TOK_INVALID:
+					wprintf( L"Type: INVALID\n" );
+					break;
+				case TOK_STRING:
+					wprintf( L"Type: STRING\t Value: %ls\n", tok_last( &tok ) );
+					break;
+				case TOK_PIPE:
+					wprintf( L"Type: PIPE\n" );
+					break;
+				case TOK_END:
+					wprintf( L"Type: END\n" );
+					break;
+				case TOK_ERROR:
+					wprintf( L"Type: ERROR\n" );
+					break;
+				default:
+					wprintf( L"Type: Unknown\n"  );
+					break;
+			}
+		}
+		tok_destroy( &tok );
+
+	}
+}
+
+
+#endif

tokenizer.h

@@ -0,0 +1,146 @@
+/** \file tokenizer.h 
+
+    A specialized tokenizer for tokenizing the fish language. In the
+    future, the tokenizer should be extended to support marks,
+    tokenizing multiple strings and disposing of unused string
+    segments.
+*/
+
+/**
+   Token types
+*/
+enum token_type
+{
+	TOK_NONE, /**< Tokenizer not yet constructed */
+	TOK_ERROR, /**< Error reading token */
+	TOK_INVALID,/**< Invalid token */
+	TOK_STRING,/**< String token */
+	TOK_PIPE,/**< Pipe token */
+	TOK_END,/**< End token */
+	TOK_REDIRECT_OUT, /**< redirection token */
+	TOK_REDIRECT_APPEND,/**< redirection append token */
+	TOK_REDIRECT_IN,/**< input redirection token */
+	TOK_REDIRECT_FD,/**< redirection to new fd token */
+	TOK_BACKGROUND,/**< send job to bg token */
+	TOK_COMMENT/**< comment token */
+}
+;
+
+/**
+   Flag telling the tokenizer to accept incomplete parameters,
+   i.e. parameters with mismatching paranthesis, etc. This is useful
+   for tab-completion.
+*/
+#define TOK_ACCEPT_UNFINISHED 1
+
+/**
+   Flag telling the tokenizer not to remove comments. Useful for
+   syntax highlighting.
+*/
+#define TOK_SHOW_COMMENTS 2
+
+
+/**
+   The tokenizer struct. 
+*/
+typedef struct
+{
+	/** A pointer into the original string, showing where the next token begins */
+	wchar_t *buff;
+	/** A copy of the original string */
+	wchar_t *orig_buff;
+	/** A pointer to the last token*/
+	wchar_t *last;
+	
+	/** Type of last token*/
+	int last_type;
+	/** Length of last token*/
+	int last_len;
+	/** Offset of last token*/
+	int last_pos;
+	/** Whether there are more tokens*/
+	int has_next;
+	/** Whether incomplete tokens are accepted*/
+	int accept_unfinished;
+	/** Whether commants should be returned*/
+	int show_comments;
+	/** Type of last quote, can be either ' or ".*/
+	wchar_t last_quote;
+}
+tokenizer;
+
+/**
+  Initialize the tokenizer. b is the string that is to be
+  tokenized. It is not copied, and should not be freed by the caller
+  until after the tokenizer is destroyed.
+
+  \param tok The tokenizer to initialize
+  \param b The string to tokenize
+  \param flags Flags to the tokenizer. Setting TOK_ACCEPT_UNFINISHED will cause the tokenizer
+  to accept incomplete tokens, such as a subshell without a closing
+  parenthesis, as a valid token. Setting TOK_SHOW_COMMENTS will return comments as tokens
+  
+*/
+void tok_init( tokenizer *tok, const wchar_t *b, int flags );
+
+/**
+  Jump to the next token.
+*/
+void tok_next( tokenizer *tok );
+
+/**
+  Returns the type of the last token. Must be one of the values in the token_type enum.
+*/
+int tok_last_type( tokenizer *tok );
+
+/**
+  Returns the last token string. The string should not be freed by the caller.
+*/
+wchar_t *tok_last( tokenizer *tok );
+
+/**
+  Returns the type of quote from the last TOK_QSTRING
+*/
+wchar_t tok_last_quote( tokenizer *tok );
+
+/**
+  Returns true as long as there are more tokens left
+*/
+int tok_has_next( tokenizer *tok );
+
+/**
+  Returns the position of the beginning of the current token in the original string
+*/
+int tok_get_pos( tokenizer *tok );
+
+/**
+   Destroy the tokenizer and free asociated memory
+*/
+void tok_destroy( tokenizer *tok );
+
+
+/**
+   Returns the original string to tokenizer
+ */
+wchar_t *tok_string( tokenizer *tok );
+
+
+/**
+   Returns only the first token from the specified string. This is a
+   convenience function, used to retrieve the first token of a
+   string. This can be useful for error messages, etc.
+
+   The string should be freed. After use.
+*/
+wchar_t *tok_first( const wchar_t *str );
+
+/**
+   Move tokenizer position
+*/
+void tok_set_pos( tokenizer *tok, int pos );
+
+/**
+   Returns a string description of the specified token type
+*/
+const wchar_t *tok_get_desc( int type );
+

user_doc.head.html

@@ -0,0 +1,25 @@
+<html>
+<head>
+<title>fish user documentation</title>
+<link href="doxygen.css" rel="stylesheet" type="text/css">
+</head>
+<body>
+
+<div class="qindex">
+
+ <a class="qindex" href="http://roo.no-ip.org/fish/index.html"><tt>fish</tt> home</a> 
+|
+ <a class="qindex" href="index.html">Main documentation page</a> 
+|
+<a class="qindex" href="about.html">About fish</a>
+|
+<a class="qindex" href="commands.html">External commands</a>
+|
+<a class="qindex" href="difference.html">How fish differs from other shells</a>
+|
+<a class="qindex" href="builtins.html">Builtin commands</a>
+|
+<a class="qindex" href="license.html">License</a>
+ 
+</div>
+

util.c

@@ -0,0 +1,986 @@
+/** \file util.c
+	Generic utilities library.
+
+	Contains datastructures such as hash tables, automatically growing array lists, priority queues, etc.
+*/
+
+#include "config.h"
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <wchar.h>
+#include <math.h>
+#include <sys/time.h>
+#include <stdarg.h>
+#include <string.h>
+#include <ctype.h>
+#include <wctype.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <dirent.h>
+#include <errno.h>
+
+
+#include "util.h"
+#include "common.h"
+#include "wutil.h"
+
+/**
+	Minimum allocated size for data structures. Used to avoid excessive
+	memory allocations for lists, hash tables, etc, which are nearly
+	empty.
+*/
+#define MIN_SIZE 128
+
+/**
+   Maximum number of characters that can be inserted using a single
+   call to sb_printf. This is needed since vswprintf doesn't tell us
+   what went wrong. We don't know if we ran out of space or something
+   else went wrong. Therefore we assume that any error is an out of
+   memory-error and try again until we reach this size.
+*/
+#define SB_MAX_SIZE 32767
+
+float minf( float a,
+			float b )
+{
+	return a<b?a:b;
+}
+
+
+float maxf( float a,
+			float b )
+{
+	return a>b?a:b;
+}
+
+int mini( int a,
+		  int b )
+{
+	return a<b?a:b;
+}
+
+
+int maxi( int a,
+		  int b )
+{
+	return a>b?a:b;
+}
+
+
+/* Queue functions */
+
+
+void q_init( queue_t *q )
+{
+	q->start = (void **)malloc( sizeof(void*)*1 );
+	q->stop = &q->start[1];
+	q->put_pos = q->get_pos = q->start;
+}
+
+void q_destroy( queue_t *q )
+{
+	free( q->start );
+}
+
+/*
+  static q_print( queue_t *q )
+  {
+  int i;
+  int size = (q->stop-q->start);
+
+  printf( "Storlek: %d\n", size );
+  for( i=0; i< size; i++ )
+  {
+  printf( " %c%c %d: %d\n",
+  &q->start[i]==q->get_pos?'g':' ',
+  &q->start[i]==q->put_pos?'p':' ',
+  i,
+  q->start[i] );
+  }
+  }
+
+*/
+
+/**
+   Reallocate the queue_t
+*/
+static int q_realloc( queue_t *q )
+{
+	void **old_start = q->start;
+	void **old_stop = q->stop;
+	int diff;
+	int new_size;
+	
+	new_size = 2*(q->stop-q->start);
+	
+	q->start=(void**)realloc( q->start, sizeof(void*)*new_size );
+	if( q->start == 0 )
+	{
+		q->start = old_start;
+		return 0;
+	}
+	
+	diff = q->start - old_start;
+	q->get_pos += diff;
+	q->stop = &q->start[new_size];
+	memcpy( old_stop + diff, q->start, sizeof(void*)*(q->get_pos-q->start));
+	q->put_pos = old_stop + diff + (q->get_pos-q->start);
+	
+	return 1;
+}
+
+int q_put( queue_t *q, void *e )
+{
+	*q->put_pos = e;
+
+//	fprintf( stderr, "Put element %d to queue %d\n", e, q );
+	
+	if( ++q->put_pos == q->stop )
+		q->put_pos = q->start;
+	if( q->put_pos == q->get_pos )
+		return q_realloc( q );
+	return 1;
+}
+
+void *q_get( queue_t *q)
+{
+	void *e = *q->get_pos;
+	if( ++q->get_pos == q->stop )
+		q->get_pos = q->start;
+	return e;
+}
+
+void *q_peek( queue_t *q )
+{
+	return *q->get_pos;
+}
+ 
+int q_empty( queue_t *q )
+{
+//	fprintf( stderr, "Queue %d is %s\n", q, (q->put_pos == q->get_pos)?"empty":"non-empty" );
+	return q->put_pos == q->get_pos;
+}
+
+/* Stack functions */
+
+
+
+
+/* Hash table functions */
+
+void hash_init2( hash_table_t *h,
+				int (*hash_func)(const void *key),
+				 int (*compare_func)(const void *key1, const void *key2),
+				 size_t capacity)
+{
+	int i;
+	size_t sz = capacity*4/3+1;
+	
+	h->arr = malloc( sizeof(hash_struct_t)*sz );
+	h->size = sz;
+	for( i=0; i< sz; i++ )
+		h->arr[i].key = 0;
+	h->count=0;
+	h->hash_func = hash_func;
+	h->compare_func = compare_func;
+}
+
+void hash_init( hash_table_t *h,
+				int (*hash_func)(const void *key),
+				int (*compare_func)(const void *key1, const void *key2) )
+{
+	hash_init2( h, hash_func, compare_func, 31 );	
+}
+
+
+void hash_destroy( hash_table_t *h )
+{
+	free( h->arr );
+}
+
+/**
+   Search for the specified hash key in the table
+   \return index in the table, or to the first free index if the key is not in the table
+*/
+static int hash_search( hash_table_t *h,
+						const void *key )
+{
+	int hv = h->hash_func( key );
+	int pos = abs(hv) % h->size;
+	while(1)
+	{
+		if( (h->arr[pos].key == 0 ) ||
+			( h->compare_func( h->arr[pos].key, key ) ) )
+		{
+			return pos;
+		}
+		pos++;
+		pos %= h->size;
+	}
+}
+
+/**
+   Reallocate the hash array. This is quite expensive, as every single entry has to be rehashed and moved.
+*/
+static int hash_realloc( hash_table_t *h,
+						 int sz )
+{
+
+	/* Avoid reallocating when using pathetically small tables */
+	if( ( sz < h->size ) && (h->size < MIN_SIZE))
+		return 1;
+	sz = maxi( sz, MIN_SIZE );
+
+	hash_struct_t *old_arr = h->arr;
+	int old_size = h->size;
+
+	int i;
+
+	h->arr = malloc( sizeof( hash_struct_t) * sz );
+	if( h->arr == 0 )
+	{
+		h->arr = old_arr;
+		return 0;
+	}
+
+	memset( h->arr,
+			0,
+			sizeof( hash_struct_t) * sz );
+	h->size = sz;
+
+	for( i=0; i<old_size; i++ )
+	{
+		if( old_arr[i].key != 0 )
+		{
+			int pos = hash_search( h, old_arr[i].key );
+			h->arr[pos].key = old_arr[i].key;
+			h->arr[pos].data = old_arr[i].data;
+		}
+	}
+	free( old_arr );
+
+	return 1;
+}
+
+
+int hash_put( hash_table_t *h,
+			  const void *key,
+			  const void *data )
+{
+	int pos;
+	
+	if( (float)(h->count+1)/h->size > 0.75f )
+	{
+		if( !hash_realloc( h, (h->size+1) * 2 -1 ) )
+		{
+			return 0;
+		}
+	}
+
+	pos = hash_search( h, key );
+
+	if( h->arr[pos].key == 0 )
+	{
+		h->count++;
+	}
+
+	h->arr[pos].key = key;
+	h->arr[pos].data = data;
+	return 1;
+}
+
+const void *hash_get( hash_table_t *h,
+					  const void *key )
+{
+	int pos = hash_search( h, key );
+	if( h->arr[pos].key == 0 )
+		return 0;
+	else
+		return h->arr[pos].data;
+}
+
+const void *hash_get_key( hash_table_t *h,
+						  const void *key )
+{
+	int pos = hash_search( h, key );
+	if( h->arr[pos].key == 0 )
+		return 0;
+	else
+		return h->arr[pos].key;
+}
+
+int hash_get_count( hash_table_t *h)
+{
+	return h->count;
+}
+
+void hash_remove( hash_table_t *h,
+				  const void *key,
+				  const void **old_key,
+				  const void **old_val )
+{
+	int pos = hash_search( h, key );
+	int next_pos;
+
+	if( h->arr[pos].key == 0 )
+	{
+
+		if( old_key != 0 )
+			*old_key = 0;
+		if( old_val != 0 )
+			*old_val = 0;
+		return;
+	}
+
+	h->count--;
+
+	if( old_key != 0 )
+		*old_key = h->arr[pos].key;
+	if( old_val != 0 )
+		*old_val = h->arr[pos].data;
+
+	h->arr[pos].key = 0;
+
+	next_pos = pos+1;
+	next_pos %= h->size;
+
+	while( h->arr[next_pos].key != 0 )
+	{
+
+		int hv = h->hash_func( h->arr[next_pos].key );
+		int ideal_pos = abs( hv ) % h->size;
+		int dist_old = (next_pos - ideal_pos + h->size)%h->size;
+		int dist_new = (pos - ideal_pos + h->size)%h->size;
+		if ( dist_new < dist_old )
+		{
+			h->arr[pos].key = h->arr[next_pos].key;
+			h->arr[pos].data = h->arr[next_pos].data;
+			h->arr[next_pos].key = 0;
+			pos = next_pos;
+		}
+		next_pos++;
+
+		next_pos %= h->size;
+
+	}
+
+	if( (float)(h->count+1)/h->size < 0.2f && h->count < 63 )
+	{
+		hash_realloc( h, (h->size+1) / 2 -1 );
+	}
+
+	return;
+}
+
+int hash_contains( hash_table_t *h,
+				   const void *key )
+{
+	int pos = hash_search( h, key );
+	return h->arr[pos].key != 0;
+}
+
+/**
+   Push hash value into array_list_t
+*/
+static void hash_put_data( const void *key,
+						   const void *data,
+						   void *al )
+{
+	al_push( (array_list_t *)al,
+			 data );
+}
+
+
+void hash_get_data( hash_table_t *h,
+					array_list_t *arr )
+{
+	hash_foreach2( h, &hash_put_data, arr );
+}
+
+/**
+   Push hash key into array_list_t
+*/
+static void hash_put_key( const void *key, const void *data, void *al )
+{
+	al_push( (array_list_t *)al, key );
+}
+
+
+void hash_get_keys( hash_table_t *h,
+					array_list_t *arr )
+{
+	hash_foreach2( h, &hash_put_key, arr );
+}
+
+void hash_foreach( hash_table_t *h,
+				   void (*func)(const void *, const void *) )
+{
+	int i;
+	for( i=0; i<h->size; i++ )
+	{
+		if( h->arr[i].key != 0 )
+		{
+			func( h->arr[i].key, h->arr[i].data );
+		}
+	}
+}
+
+void hash_foreach2( hash_table_t *h,
+					void (*func)( const void *, const void *, void * ),
+					void *aux )
+{
+	int i;
+	for( i=0; i<h->size; i++ )
+	{
+		if( h->arr[i].key != 0 )
+		{
+			func( h->arr[i].key, h->arr[i].data, aux );
+		}
+	}
+}
+
+
+int hash_str_cmp( const void *a, const void *b )
+{
+	return strcmp((char *)a,(char *)b) == 0;
+}
+
+/**
+   Helper function for hash_wcs_func
+*/
+static uint rotl5( uint in )
+{
+	return (in<<5|in>>27);
+}
+
+
+int hash_str_func( const void *data )
+{
+	int res = 0x67452301u;
+	const char *str = data;	
+
+	while( *str )
+		res = (18499*rotl5(res)) ^ *str++;
+	
+	return res;
+}
+
+int hash_wcs_func( const void *data )
+{
+	int res = 0x67452301u;
+	const wchar_t *str = data;	
+
+	while( *str )
+		res = (18499*rotl5(res)) ^ *str++;
+	
+	return res;
+}
+
+
+int hash_wcs_cmp( const void *a, const void *b )
+{
+	return wcscmp((wchar_t *)a,(wchar_t *)b) == 0;
+}
+
+void pq_init( priority_queue_t *q,
+			  int (*compare)(void *e1, void *e2) )
+{
+	q->arr=0;
+	q->size=0;
+	q->count=0;
+	q->compare = compare;
+}
+
+/**
+   Check that the priority queue is in a valid state
+*/
+/*
+  static void pq_check( priority_queue_t *q, int i )
+  {
+  int l,r;
+  if( q->count <= i )
+  return;
+
+  l=i*2+1;
+  r=i*2+2;
+
+
+  if( (q->count > l) && (q->compare(q->arr[i], q->arr[l]) < 0) )
+  {
+  printf( "ERROR: Place %d less than %d\n", i, l );
+  }
+  if( (q->count > r) && (q->compare(q->arr[i], q->arr[r]) < 0) )
+  {
+  printf( "ERROR: Place %d less than %d\n", i, r );
+  }
+  pq_check( q, l );
+  pq_check( q, r );
+  }
+*/
+
+int pq_put( priority_queue_t *q,
+			void *e )
+{
+	int i;
+
+	if( q->size == q->count )
+	{
+		void **old_arr = q->arr;
+		int old_size = q->size;
+		q->size = maxi( 4, 2*q->size );
+		q->arr = (void **)realloc( q->arr, sizeof(void*)*q->size );
+		if( q->arr == 0 )
+		{
+			q->arr = old_arr;
+			q->size = old_size;
+			return 0;
+		}
+	}
+
+	i = q->count;
+	while( (i>0) && (q->compare( q->arr[(i-1)/2], e )<0 ) )
+	{
+		q->arr[i] = q->arr[(i-1)/2];
+		i = (i-1)/2;
+	}
+	q->arr[i]=e;
+
+	q->count++;
+
+	return 1;
+
+}
+
+/**
+   Make a valid head
+*/
+static void pq_heapify( priority_queue_t *q, int i )
+{
+	int l, r, largest;
+	l = 2*(i)+1;
+	r = 2*(i)+2;
+	if( (l < q->count) && (q->compare(q->arr[l],q->arr[i])>0) )
+	{
+		largest = l;
+	}
+	else
+	{
+		largest = i;
+	}
+	if( (r < q->count) && (q->compare( q->arr[r],q->arr[largest])>0) )
+	{
+		largest = r;
+	}
+
+	if( largest != i )
+	{
+		void *tmp = q->arr[largest];
+		q->arr[largest]=q->arr[i];
+		q->arr[i]=tmp;
+		pq_heapify( q, largest );
+	}
+}
+
+void *pq_get( priority_queue_t *q )
+{
+	void *result = q->arr[0];
+	q->arr[0] = q->arr[--q->count];
+	pq_heapify( q, 0 );
+
+/*	pq_check(q, 0 );	*/
+/*	pq_print( q ); */
+
+	return result;
+}
+
+void *pq_peek( priority_queue_t *q )
+{
+	return q->arr[0];
+}
+
+int pq_empty( priority_queue_t *q )
+{
+	return q->count == 0;
+}
+
+int pq_get_count( priority_queue_t *q )
+{
+	return q->count;
+}
+
+void pq_destroy(  priority_queue_t *q )
+{
+	free( q->arr );
+}
+
+
+void al_init( array_list_t *l )
+{
+	memset( l, 0, sizeof( array_list_t ) );
+}
+
+void al_destroy( array_list_t *l )
+{
+	free( l->arr );
+}
+
+int al_push( array_list_t *l, const void *o )
+{
+	if( l->pos >= l->size )
+	{
+		int new_size = l->pos == 0 ? MIN_SIZE : 2 * l->pos;
+		void *tmp = realloc( l->arr, sizeof( void *)*new_size );
+		if( tmp == 0 )
+			return 0;
+		l->arr = tmp;
+		l->size = new_size;		
+	}
+	l->arr[l->pos++] = o;
+	return 1;
+}
+
+int al_push_all( array_list_t *a, array_list_t *b )
+{
+	int k;
+	for( k=0; k<al_get_count( b ); k++ )
+	{
+		if( !al_push( a, al_get( b, k ) ) )
+			return 0;
+	}
+	return 1;
+}
+
+
+int al_set( array_list_t *l, int pos, const void *o )
+{
+	int old_pos;
+	
+	if( pos < 0 )
+		return 0;
+	if( pos < l->pos )
+	{
+		l->arr[pos] = o;
+		return 1;
+	}
+	old_pos=l->pos;
+	
+	l->pos = pos;
+	if( al_push( l, o ) )
+	{
+/*		fwprintf( stderr, L"Clearing from index %d to index %d\n", 
+				  old_pos, pos );
+*/	
+		memset( &l->arr[old_pos], 
+				0,
+				sizeof(void *) * (pos - old_pos) );
+		return 1;		
+	}
+	return 0;	
+}
+
+const void *al_get( array_list_t *l, int pos )
+{
+	if( pos < 0 )
+		return 0;
+	if( pos >= l->pos )
+		return 0;
+	return l->arr[pos];
+}
+
+void al_truncate( array_list_t *l, int new_sz )
+{
+	l->pos = new_sz;
+}
+
+const void *al_pop( array_list_t *l )
+{
+	const void *e = l->arr[--l->pos];
+	if( (l->pos*3 < l->size) && (l->size < MIN_SIZE) )
+	{
+		const void ** old_arr = l->arr;
+		int old_size = l->size;
+		l->size = l->size/2;
+		l->arr = realloc( l->arr, sizeof(void*)*l->size );
+		if( l->arr == 0 )
+		{
+			l->arr = old_arr;
+			l->size = old_size;
+		}
+	}
+	return e;
+}
+
+const void *al_peek( array_list_t *l )
+{
+
+	return l->pos>0?l->arr[l->pos-1]:0;
+}
+
+int al_empty( array_list_t *l )
+{
+	return l->pos == 0;
+}
+
+int al_get_count( array_list_t *l )
+
+{
+	return l->pos;
+}
+
+void al_foreach( array_list_t *l, void (*func)( const void * ))
+{
+	int i;
+	for( i=0; i<l->pos; i++ )
+		func( l->arr[i] );
+}
+
+void al_foreach2( array_list_t *l, void (*func)( const void *, void *), void *aux)
+{
+	int i;
+	for( i=0; i<l->pos; i++ )
+		func( l->arr[i], aux );
+}
+
+int wcsfilecmp( const wchar_t *a, const wchar_t *b )
+{
+	if( *a==0 )
+	{
+		if( *b==0)
+			return 0;
+		return -1;
+	}
+	if( *b==0 )
+	{
+		return 1;
+	}
+	int secondary_diff=0;
+	if( iswdigit( *a ) && iswdigit( *b ) )
+	{
+		wchar_t *aend, *bend;
+		long al = wcstol( a, &aend, 10 );
+		long bl = wcstol( b, &bend, 10 );
+		int diff = al - bl;
+		if( diff )
+			return diff>0?2:-2;
+
+		secondary_diff = (aend-a) - (bend-b);
+
+		a=aend-1;
+		b=bend-1;
+	}
+	else
+	{
+		int diff = towlower(*a) - towlower(*b);
+		if( diff != 0 )
+			return (diff>0)?2:-2;
+
+		secondary_diff = *a-*b;
+	}
+
+	int res = wcsfilecmp( a+1, b+1 );
+	switch( abs(res) )
+	{
+		case 2:
+			return res;
+		default:
+			if( secondary_diff )
+				return secondary_diff>0?1:-1;
+	}
+	return 0;
+
+}
+
+void sb_init( string_buffer_t * b)
+{
+	wchar_t c=0;
+	memset( b, 0, sizeof(string_buffer_t) );	
+	b_append( b, &c, sizeof( wchar_t));
+	b->used -= sizeof(wchar_t);
+}
+
+void sb_append( string_buffer_t *b, const wchar_t * s)
+{
+//	fwprintf( stderr, L"Append string \'%ls\'\n", s );
+
+	if( !s )
+		return;
+
+
+	b_append( b, s, sizeof(wchar_t)*(wcslen(s)+1) );
+	b->used -= sizeof(wchar_t);
+}
+
+void sb_append_substring( string_buffer_t *b, const wchar_t *s, size_t l )
+{
+    wchar_t tmp=0;
+
+	if( !s )
+		return;
+
+    b_append( b, s, sizeof(wchar_t)*l );
+    b_append( b, &tmp, sizeof(wchar_t) );
+    b->used -= sizeof(wchar_t);
+}
+
+
+void sb_append_char( string_buffer_t *b, wchar_t c )
+{
+	wchar_t buff[2]=
+		{
+			c, 0
+		}
+	;
+	sb_append( b, buff );
+
+}
+
+void sb_append2( string_buffer_t *b, ... )
+{
+	va_list va;
+	wchar_t *arg;
+
+	va_start( va, b );
+	while( (arg=va_arg(va, wchar_t *) )!= 0 )
+	{
+		sb_append( b, arg );
+	}
+	va_end( va );
+}
+
+int sb_printf( string_buffer_t *buffer, const wchar_t *format, ... )
+{
+	va_list va;
+	int res;
+	
+	if( !buffer->length )
+	{
+		buffer->length = MIN_SIZE;
+		buffer->buff = malloc( MIN_SIZE );
+		if( !buffer->buff )
+			die_mem();
+	}
+	
+
+	while( 1 )
+	{
+		
+		va_start( va, format );
+		res = vswprintf( (wchar_t *)((char *)buffer->buff+buffer->used), 
+						 (buffer->length-buffer->used)/sizeof(wchar_t), 
+						 format,
+						 va );
+		
+		if( res >= 0 )
+		{
+			buffer->used+= res*sizeof(wchar_t);
+			break;
+		}
+
+		/*
+		  As far as I know, there is no way to check if a
+		  vswprintf-call failed because of a badly formated string
+		  option or because the supplied destination string was to
+		  small. In GLIBC, errno seems to be set to EINVAL either way. 
+
+		  Because of this, sb_printf will on failiure try to
+		  increase the buffer size until the free space is larger than
+		  SB_MAX_SIZE, at which point it will conclude that the error
+		  was probably due to a badly formated string option, and
+		  return an error. 
+		*/
+	
+		if( buffer->length - buffer->used > SB_MAX_SIZE )
+			break;
+
+		buffer->buff = realloc( buffer->buff, 2*buffer->length );
+		if( !buffer->buff )
+			die_mem();
+		buffer->length *= 2;				
+	}
+	va_end( va );	
+	return res;	
+}
+
+
+
+
+void sb_destroy( string_buffer_t * b )
+{
+	free( b->buff );
+}
+
+void sb_clear( string_buffer_t * b )
+{
+	free( b->buff );
+	sb_init( b );
+}
+
+
+void b_init( buffer_t *b)
+{
+	memset( b,0,sizeof(buffer_t));
+}
+
+
+
+void b_destroy( buffer_t *b )
+{
+	free( b->buff );
+}
+
+
+void b_append( buffer_t *b, const void *d, ssize_t len )
+{
+	if( len<=0 )
+		return;
+
+	if( !b )
+	{
+		debug( 2, L"Copy to null buffer" );
+		return;
+	}
+
+	if( !d )
+	{
+		debug( 2, L"Copy from null pointer" );
+		return;
+	}
+
+	if( len < 0 )
+	{
+		debug( 2, L"Negative number of characters to be copied" );
+		return;
+	}
+
+
+	if( b->length <= (b->used + len) )
+	{
+		size_t l = maxi( b->length*2,
+						 maxi( b->used+len+MIN_SIZE,MIN_SIZE));
+
+		void *d = realloc( b->buff, l );
+		if( !d )
+		{
+			die_mem();
+			
+		}
+		b->buff=d;
+		b->length = l;
+	}
+	memcpy( ((char*)b->buff)+b->used,
+			d,
+			len );
+
+//	fwprintf( stderr, L"Copy %s, new value %s\n", d, b->buff );
+	b->used+=len;
+}
+
+long long get_time()
+{
+	struct timeval time_struct;
+	gettimeofday( &time_struct, 0 );
+	return 1000000ll*time_struct.tv_sec+time_struct.tv_usec;
+}
+

util.h

@@ -0,0 +1,463 @@
+/** \file util.h
+	Generic utilities library.
+*/
+
+/**
+   Data structure for an automatically resizing dynamically allocated queue,
+*/
+typedef struct queue
+{
+	/** Start of the array */
+	void **start;
+	/** End of the array*/
+	void **stop;
+	/** Where to insert elements */
+	void **put_pos;
+	/** Where to remove elements */
+	void **get_pos;
+}
+queue_t;
+
+/**
+   Internal struct used by hash_table_t.
+*/
+typedef struct
+{
+	/** Hash key*/
+	const void *key;
+	/** Value */
+	const void *data;
+}
+hash_struct_t;
+
+/**
+   Data structure for the hash table implementaion. A hash table allows for
+   retrieval and removal of any element in O(1), so long as a proper
+   hash function is supplied.
+
+   The hash table is implemented using a single hash function and
+   element storage directly in the array. When a collision occurs, the
+   hashtable iterates until a zero element is found. When the table is
+   75% full, it will automatically reallocate itself. This
+   reallocation takes O(n) time. The table is guaranteed to never be
+   more than 75% full or less than 30% full (Unless the table is
+   nearly empty). Its size is always a Mersenne number.
+
+*/
+
+typedef struct hash_table
+{
+	/** The array containing the data */
+	hash_struct_t *arr;
+	/** Number of elements */
+	int count;
+	/** Length of array */
+	int size;
+	/** Hash function */
+	int (*hash_func)( const void *key );
+	/** Comparison function */
+	int (*compare_func)( const void *key1, const void *key2 );
+}
+hash_table_t;
+
+/**
+   Data structure for an automatically resizing dynamically allocated
+   priority queue. A priority queue allows quick retrieval of the
+   smallest element of a set (This implementation uses O(log n) time).
+   This implementation uses a heap for storing the queue.
+*/
+typedef struct priority_queue
+{
+	/** Array contining the data */
+	void **arr;
+	/** Number of elements*/
+	int count;
+	/** Length of array */
+	int size;
+	/** Comparison function */
+	int (*compare)(void *e1, void *e2);
+}
+priority_queue_t;
+
+/**
+   Array list struct.
+   A dynamically growing list that supports stack operations.
+*/
+typedef struct array_list
+{
+	/** Array containing the data */
+	const void **arr;
+	/** Position to append elements at*/
+	int pos;
+	/** Length of array */
+	int size;
+}
+array_list_t;
+
+/**
+   Linked list node.
+*/
+typedef struct _ll_node
+{
+	/** Next node */
+	struct _ll_node *next, /** Previous node */ *prev;
+	/** Node data */
+	void *data;
+}
+ll_node_t;
+
+/**
+   Buffer for concatenating arbitrary data.
+*/
+typedef struct buffer
+{
+	char *buff; /**<data buffer*/
+	size_t length; /**< Size of buffer */
+	size_t used; /**< Size of data in buffer */
+}
+buffer_t;
+
+
+/**
+   String buffer struct.  An autoallocating buffer used for
+   concatenating strings. This is really just a buffer_t.
+*/
+typedef buffer_t string_buffer_t;
+
+	
+
+/**
+   Returns the larger of two ints
+*/
+int maxi( int a, int b );
+/**
+   Returns the smaller of two ints
+ */
+int mini( int a, int b );
+/**
+   Returns the larger of two floats
+*/
+float maxf( float a, float b );
+/**
+   Returns the smaller of two floats
+*/
+float minf( float a, float b );
+
+/*
+  All the datastuctures below autoresize. The queue, stack and
+  priority queue are all impemented using an array and are guaranteed
+  to never be less than 50% full. 
+*/
+
+/** 
+	Initialize the queue. A queue is a FIFO buffer, i.e. the first
+    element to be inserted into the buffer is the first element to be
+    returned. 
+*/
+void q_init( queue_t *q );
+/** Destroy the queue */
+void q_destroy( queue_t *q );
+/** Insert element into queue */
+int q_put( queue_t *q, void *e );
+/** Remove and return next element from queue */
+void *q_get( queue_t *q);
+/** Return next element from queue without removing it */
+void *q_peek( queue_t *q);
+/** Returns 1 if the queue is empty, 0 otherwise */
+int q_empty( queue_t *q );
+
+/**
+   Initialize a hash table. The hash function must never return the value 0.
+*/
+void hash_init( hash_table_t *h,
+				int (*hash_func)(const void *key),
+				int (*compare_func)(const void *key1, const void *key2) );
+
+/**
+   Initialize a hash table. The hash function must never return the value 0.
+*/
+void hash_init2( hash_table_t *h,
+				int (*hash_func)(const void *key),
+				 int (*compare_func)(const void *key1, const void *key2),
+				 size_t capacity);
+
+/**
+   Destroy the hash table and free associated memory.
+*/
+void hash_destroy( hash_table_t *h );
+/**
+   Set the key/value pair for the hashtable. 
+*/
+int hash_put( hash_table_t *h, 
+			  const void *key,
+			  const void *data );
+/**
+   Returns the data with the associated key, or 0 if no such key is in the hashtable
+*/
+const void *hash_get( hash_table_t *h,
+					  const void *key );
+/**
+   Returns the hash tables version of the specified key
+*/
+const void *hash_get_key( hash_table_t *h, 
+						  const void *key );
+
+/**
+   Returns the number of key/data pairs in the table.
+*/
+int hash_get_count( hash_table_t *h);
+/**
+   Remove the specified key from the hash table
+
+   \param h The hashtable
+   \param key The key
+   \param old_key If not 0, a pointer to the old key will be stored at the specified address
+   \param old_data If not 0, a pointer to the data will be stored at the specified address
+*/
+void hash_remove( hash_table_t *h, 
+				  const void *key, 
+				  const void **old_key,
+				  const void **old_data );
+/**
+   Checks whether the specified key is in the hash table
+*/
+int hash_contains( hash_table_t *h, 
+				   const void *key );
+
+/**
+   Appends all keys in the table to the specified list
+*/
+void hash_get_keys( hash_table_t *h,
+					array_list_t *arr );
+/**
+   Appends all data elements in the table to the specified list
+*/
+void hash_get_data( hash_table_t *h,
+					array_list_t *arr );
+/** Call the function func for each key/data pair in the table*/
+void hash_foreach( hash_table_t *h, 
+				   void (*func)( const void *, const void * ) );
+/** Same as hash_foreach, but the function func takes an additional
+ * argument, which is provided by the caller in the variable aux */
+void hash_foreach2( hash_table_t *h, void (*func)( const void *, 
+												 const void *, 
+												 void *), 
+					void *aux );
+
+/**
+   Hash function suitable for character strings. 
+*/
+int hash_str_func( const void *data );
+/**
+   Hash comparison function suitable for character strings
+*/
+int hash_str_cmp( const void *a, const void *b );
+
+/**
+   Hash function suitable for wide character strings. 
+*/
+int hash_wcs_func( const void *data );
+
+/**
+   Hash comparison function suitable for wide character strings
+*/
+int hash_wcs_cmp( const void *a, const void *b );
+
+/** Initialize the priority queue
+
+ \param q the queue to initialize
+ \param compare a comparison function that can compare two entries in the queue
+*/
+void pq_init( priority_queue_t *q,
+			  int (*compare)(void *e1, void *e2) );
+/**
+   Add element to the queue
+
+ \param q the queue
+ \param e the new element
+ 
+*/
+int pq_put( priority_queue_t *q,
+			 void *e );
+/**
+  Removes and returns the last entry in the priority queue
+*/
+void *pq_get( priority_queue_t *q );
+/**
+  Returns the last entry in the priority queue witout removing it.
+*/
+void *pq_peek( priority_queue_t *q );
+
+/** 
+	Returns 1 if the priority queue is empty, 0 otherwise.
+*/
+int pq_empty( priority_queue_t *q );
+
+/**
+   Returns the number of elements in the priority queue.
+*/
+int pq_get_count( priority_queue_t *q );
+
+/** 
+	Destroy the priority queue and free memory used by it.
+*/
+void pq_destroy(  priority_queue_t *q );
+
+/** Initialize the list. */
+void al_init( array_list_t *l );
+/** Destroy the list and free memory used by it.*/
+void al_destroy( array_list_t *l );
+/**
+   Append element to list
+
+   \param l The list
+   \param o The element
+   \return
+   \return 1 if succesfull, 0 otherwise
+*/
+int al_push( array_list_t *l, const void *o );
+/**
+   Append all elements of a list to another
+
+   \param a The destination list
+   \param b The source list
+   \return 1 if succesfull, 0 otherwise
+*/
+int al_push_all( array_list_t *a, array_list_t *b );
+/**
+   Sets the element at the specified index
+
+   \param l The array_list_t
+   \param pos The index 
+   \param o The element 
+*/
+int al_set( array_list_t *l, int pos, const void *o );
+/**
+   Returns the element at the specified index
+
+   \param l The array_list_t
+   \param pos The index 
+   \return The element 
+*/
+const void *al_get( array_list_t *l, int pos );
+/**
+  Truncates the list to new_sz items.
+*/
+void al_truncate( array_list_t *l, int new_sz );
+/**
+  Removes and returns the last entry in the list
+*/
+const void *al_pop( array_list_t *l );
+/**
+   Returns the number of elements in the list
+*/
+int al_get_count( array_list_t *l );
+/**
+  Returns the last entry in the list witout removing it.
+*/
+const void *al_peek( array_list_t *l );
+/** Returns 1 if the list is empty, 0 otherwise*/
+int al_empty( array_list_t *l);
+/** Call the function func for each entry in the list*/
+void al_foreach( array_list_t *l, void (*func)(const void * ));
+/** 
+	Same as al_foreach, but the function func takes an additional
+	argument, which is provided by the caller in the variable aux
+*/
+void al_foreach2( array_list_t *l, void (*func)(const void *, void *), void *aux);
+
+/**
+   Compares two wide character strings without case but with  
+   a logical ordering for numbers.
+
+   This function tries to order strings in a way which is intuitive to
+   humans with regards to sorting strings containing numbers.
+
+   Most sorting functions would sort the strings 'file1.txt'
+   'file5.txt' and 'file12.txt' as:
+
+   file1.txt
+   file12.txt
+   file5.txt
+
+   This function regards any sequence of digits as a single entity
+   when performing comparisons, so the output is instead:
+
+   file1.txt
+   file5.txt
+   file12.txt
+
+   Which most people would find more intuitive.
+
+   The system breaks down if the user is using numbers of a base larger than 10.
+*/
+int wcsfilecmp( const wchar_t *a, const wchar_t *b );
+
+
+/**
+   Initialize the specified string_buffer
+*/
+void sb_init( string_buffer_t * );
+
+/**
+   Append a string to the buffer
+*/
+void sb_append( string_buffer_t *, const wchar_t * );
+
+/**
+   Append a part of a string to the buffer
+*/
+void sb_append_substring( string_buffer_t *, const wchar_t *, size_t );
+
+/**
+   Append a character to the buffer
+*/
+void sb_append_char( string_buffer_t *, wchar_t );
+
+
+/**
+   Append a null terminated list of strings to the buffer.
+   Example:
+
+   sb_append2( my_buff, L"foo", L"bar", 0 );
+*/
+void sb_append2( string_buffer_t *, ... );
+
+/**
+   Append formated string data to the buffer. This function internally
+   relies on \c vswprintf, so any filter options supported by that
+   function is also supported by this function
+*/
+int sb_printf( string_buffer_t *buffer, const wchar_t *format, ... );
+
+
+/**
+  Destroy the buffer and free it's memory
+*/
+void sb_destroy( string_buffer_t * );
+
+/**
+   Truncate the buffer.
+*/
+void sb_clear( string_buffer_t * );
+
+
+/**
+   Initialize the specified buffer_t
+ */
+void b_init( buffer_t *b);
+/**
+   Destroy the specified buffer_t
+*/
+
+void b_destroy( buffer_t *b );
+
+/**
+   Add data of the specified length to the specified buffer_t
+*/
+void b_append( buffer_t *b, const void *d, ssize_t len );
+
+/**
+   Get the current time in microseconds since Jan 1, 1970
+*/
+long long get_time();

wgetopt.c

@@ -0,0 +1,696 @@
+/* Getopt for GNU.
+   NOTE: getopt is now part of the C library, so if you don't know what
+   "Keep this file name-space clean" means, talk to roland@gnu.ai.mit.edu
+   before changing it!
+
+   Copyright (C) 1987, 88, 89, 90, 91, 92, 93, 94
+   	Free Software Foundation, Inc.
+
+This file is part of the GNU C Library.  Its master source is NOT part of
+the C library, however.  The master source lives in /gd/gnu/lib.
+
+The GNU C Library is free software; you can redistribute it and/or
+modify it under the terms of the GNU Library General Public License as
+published by the Free Software Foundation; either version 2 of the
+License, or (at your option) any later version.
+
+The GNU C Library is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+Library General Public License for more details.
+
+You should have received a copy of the GNU Library General Public
+License along with the GNU C Library; see the file COPYING.LIB.  If
+not, write to the Free Software Foundation, Inc., 675 Mass Ave,
+Cambridge, MA 02139, USA.  */
+
+/* This tells Alpha OSF/1 not to define a getopt prototype in <stdio.h>.
+   Ditto for AIX 3.2 and <stdlib.h>.  */
+#ifndef _NO_PROTO
+#define _NO_PROTO
+#endif
+
+#include "config.h"
+
+#if !defined (__STDC__) || !__STDC__
+/* This is a separate conditional since some stdc systems
+   reject `defined (const)'.  */
+#ifndef const
+#define const
+#endif
+#endif
+
+#include <stdio.h>
+#include <wchar.h>
+
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <dirent.h>
+
+
+/* This needs to come after some library #include
+   to get __GNU_LIBRARY__ defined.  */
+#ifdef	__GNU_LIBRARY__
+/* Don't include stdlib.h for non-GNU C libraries because some of them
+   contain conflicting prototypes for getopt.  */
+#include <stdlib.h>
+#endif	/* GNU C library.  */
+
+/* This version of `getopt' appears to the caller like standard Unix `getopt'
+   but it behaves differently for the user, since it allows the user
+   to intersperse the options with the other arguments.
+
+   As `getopt' works, it permutes the elements of ARGV so that,
+   when it is done, all the options precede everything else.  Thus
+   all application programs are extended to handle flexible argument order.
+
+   Setting the environment variable POSIXLY_CORRECT disables permutation.
+   Then the behavior is completely standard.
+
+   GNU application programs can use a third alternative mode in which
+   they can distinguish the relative order of options and other arguments.  */
+
+#include "wgetopt.h"
+#include "wutil.h"
+
+
+/* For communication from `getopt' to the caller.
+   When `getopt' finds an option that takes an argument,
+   the argument value is returned here.
+   Also, when `ordering' is RETURN_IN_ORDER,
+   each non-option ARGV-element is returned here.  */
+
+wchar_t *woptarg = NULL;
+
+/* Index in ARGV of the next element to be scanned.
+   This is used for communication to and from the caller
+   and for communication between successive calls to `getopt'.
+
+   On entry to `getopt', zero means this is the first call; initialize.
+
+   When `getopt' returns EOF, this is the index of the first of the
+   non-option elements that the caller should itself scan.
+
+   Otherwise, `woptind' communicates from one call to the next
+   how much of ARGV has been scanned so far.  */
+
+/* XXX 1003.2 says this must be 1 before any call.  */
+int woptind = 0;
+
+/* The next char to be scanned in the option-element
+   in which the last option character we returned was found.
+   This allows us to pick up the scan where we left off.
+
+   If this is zero, or a null string, it means resume the scan
+   by advancing to the next ARGV-element.  */
+
+static wchar_t *nextchar;
+
+/* Callers store zero here to inhibit the error message
+   for unrecognized options.  */
+
+int wopterr = 1;
+
+/* Set to an option character which was unrecognized.
+   This must be initialized on some systems to avoid linking in the
+   system's own getopt implementation.  */
+
+int woptopt = '?';
+
+/* Describe how to deal with options that follow non-option ARGV-elements.
+
+   If the caller did not specify anything,
+   the default is REQUIRE_ORDER if the environment variable
+   POSIXLY_CORRECT is defined, PERMUTE otherwise.
+
+   REQUIRE_ORDER means don't recognize them as options;
+   stop option processing when the first non-option is seen.
+   This is what Unix does.
+   This mode of operation is selected by either setting the environment
+   variable POSIXLY_CORRECT, or using `+' as the first character
+   of the list of option characters.
+
+   PERMUTE is the default.  We permute the contents of ARGV as we scan,
+   so that eventually all the non-options are at the end.  This allows options
+   to be given in any order, even with programs that were not written to
+   expect this.
+
+   RETURN_IN_ORDER is an option available to programs that were written
+   to expect options and other ARGV-elements in any order and that care about
+   the ordering of the two.  We describe each non-option ARGV-element
+   as if it were the argument of an option with character code 1.
+   Using `-' as the first character of the list of option characters
+   selects this mode of operation.
+
+   The special argument `--' forces an end of option-scanning regardless
+   of the value of `ordering'.  In the case of RETURN_IN_ORDER, only
+   `--' can cause `getopt' to return EOF with `woptind' != ARGC.  */
+
+static enum
+{
+  REQUIRE_ORDER, PERMUTE, RETURN_IN_ORDER
+} ordering;
+
+/* Value of POSIXLY_CORRECT environment variable.  */
+static char *posixly_correct;
+
+#ifdef	__GNU_LIBRARY__
+/* We want to avoid inclusion of string.h with non-GNU libraries
+   because there are many ways it can cause trouble.
+   On some systems, it contains special magic macros that don't work
+   in GCC.  */
+#include <string.h>
+#define	my_index	wcschr
+#else
+
+/* Avoid depending on library functions or files
+   whose names are inconsistent.  */
+
+char *getenv ();
+
+static wchar_t *
+my_index (str, chr)
+     const wchar_t *str;
+     int chr;
+{
+  while (*str)
+    {
+      if (*str == chr)
+	return (wchar_t *) str;
+      str++;
+    }
+  return 0;
+}
+
+/* If using GCC, we can safely declare strlen this way.
+   If not using GCC, it is ok not to declare it.  */
+#ifdef __GNUC__
+/* Note that Motorola Delta 68k R3V7 comes with GCC but not stddef.h.
+   That was relevant to code that was here before.  */
+#if !defined (__STDC__) || !__STDC__
+/* gcc with -traditional declares the built-in strlen to return int,
+   and has done so at least since version 2.4.5. -- rms.  */
+extern int wcslen (const wchar_t *);
+#endif /* not __STDC__ */
+#endif /* __GNUC__ */
+
+#endif /* not __GNU_LIBRARY__ */
+
+/* Handle permutation of arguments.  */
+
+/* Describe the part of ARGV that contains non-options that have
+   been skipped.  `first_nonopt' is the index in ARGV of the first of them;
+   `last_nonopt' is the index after the last of them.  */
+
+static int first_nonopt;
+static int last_nonopt;
+
+/* Exchange two adjacent subsequences of ARGV.
+   One subsequence is elements [first_nonopt,last_nonopt)
+   which contains all the non-options that have been skipped so far.
+   The other is elements [last_nonopt,woptind), which contains all
+   the options processed since those non-options were skipped.
+
+   `first_nonopt' and `last_nonopt' are relocated so that they describe
+   the new indices of the non-options in ARGV after they are moved.  */
+
+static void
+exchange (argv)
+     wchar_t **argv;
+{
+  int bottom = first_nonopt;
+  int middle = last_nonopt;
+  int top = woptind;
+  wchar_t *tem;
+  
+  /* Exchange the shorter segment with the far end of the longer segment.
+     That puts the shorter segment into the right place.
+     It leaves the longer segment in the right place overall,
+     but it consists of two parts that need to be swapped next.  */
+
+  while (top > middle && middle > bottom)
+    {
+      if (top - middle > middle - bottom)
+	{
+	  /* Bottom segment is the short one.  */
+	  int len = middle - bottom;
+	  register int i;
+
+	  /* Swap it with the top part of the top segment.  */
+	  for (i = 0; i < len; i++)
+	    {
+	      tem = argv[bottom + i];
+	      argv[bottom + i] = argv[top - (middle - bottom) + i];
+	      argv[top - (middle - bottom) + i] = tem;
+	    }
+	  /* Exclude the moved bottom segment from further swapping.  */
+	  top -= len;
+	}
+      else
+	{
+	  /* Top segment is the short one.  */
+	  int len = top - middle;
+	  register int i;
+
+	  /* Swap it with the bottom part of the bottom segment.  */
+	  for (i = 0; i < len; i++)
+	    {
+	      tem = argv[bottom + i];
+	      argv[bottom + i] = argv[middle + i];
+	      argv[middle + i] = tem;
+	    }
+	  /* Exclude the moved top segment from further swapping.  */
+	  bottom += len;
+	}
+    }
+
+  /* Update records for the slots the non-options now occupy.  */
+
+  first_nonopt += (woptind - last_nonopt);
+  last_nonopt = woptind;
+}
+
+/* Initialize the internal data when the first call is made.  */
+
+static const wchar_t *
+_wgetopt_initialize (optstring)
+     const wchar_t *optstring;
+{
+  /* Start processing options with ARGV-element 1 (since ARGV-element 0
+     is the program name); the sequence of previously skipped
+     non-option ARGV-elements is empty.  */
+
+  first_nonopt = last_nonopt = woptind = 1;
+
+  nextchar = NULL;
+
+  posixly_correct = getenv ("POSIXLY_CORRECT");
+
+  /* Determine how to handle the ordering of options and nonoptions.  */
+
+  if (optstring[0] == '-')
+    {
+      ordering = RETURN_IN_ORDER;
+      ++optstring;
+    }
+  else if (optstring[0] == '+')
+    {
+      ordering = REQUIRE_ORDER;
+      ++optstring;
+    }
+  else if (posixly_correct != NULL)
+    ordering = REQUIRE_ORDER;
+  else
+    ordering = PERMUTE;
+
+  return optstring;
+}
+
+/* Scan elements of ARGV (whose length is ARGC) for option characters
+   given in OPTSTRING.
+
+   If an element of ARGV starts with '-', and is not exactly "-" or "--",
+   then it is an option element.  The characters of this element
+   (aside from the initial '-') are option characters.  If `getopt'
+   is called repeatedly, it returns successively each of the option characters
+   from each of the option elements.
+
+   If `getopt' finds another option character, it returns that character,
+   updating `woptind' and `nextchar' so that the next call to `getopt' can
+   resume the scan with the following option character or ARGV-element.
+
+   If there are no more option characters, `getopt' returns `EOF'.
+   Then `woptind' is the index in ARGV of the first ARGV-element
+   that is not an option.  (The ARGV-elements have been permuted
+   so that those that are not options now come last.)
+
+   OPTSTRING is a string containing the legitimate option characters.
+   If an option character is seen that is not listed in OPTSTRING,
+   return '?' after printing an error message.  If you set `wopterr' to
+   zero, the error message is suppressed but we still return '?'.
+
+   If a char in OPTSTRING is followed by a colon, that means it wants an arg,
+   so the following text in the same ARGV-element, or the text of the following
+   ARGV-element, is returned in `optarg'.  Two colons mean an option that
+   wants an optional arg; if there is text in the current ARGV-element,
+   it is returned in `woptarg', otherwise `woptarg' is set to zero.
+
+   If OPTSTRING starts with `-' or `+', it requests different methods of
+   handling the non-option ARGV-elements.
+   See the comments about RETURN_IN_ORDER and REQUIRE_ORDER, above.
+
+   Long-named options begin with `--' instead of `-'.
+   Their names may be abbreviated as long as the abbreviation is unique
+   or is an exact match for some defined option.  If they have an
+   argument, it follows the option name in the same ARGV-element, separated
+   from the option name by a `=', or else the in next ARGV-element.
+   When `getopt' finds a long-named option, it returns 0 if that option's
+   `flag' field is nonzero, the value of the option's `val' field
+   if the `flag' field is zero.
+
+   The elements of ARGV aren't really const, because we permute them.
+   But we pretend they're const in the prototype to be compatible
+   with other systems.
+
+   LONGOPTS is a vector of `struct option' terminated by an
+   element containing a name which is zero.
+
+   LONGIND returns the index in LONGOPT of the long-named option found.
+   It is only valid when a long-named option has been found by the most
+   recent call.
+
+   If LONG_ONLY is nonzero, '-' as well as '--' can introduce
+   long-named options.  */
+
+int
+_wgetopt_internal (argc, argv, optstring, longopts, longind, long_only)
+     int argc;
+     wchar_t *const *argv;
+     const wchar_t *optstring;
+     const struct woption *longopts;
+     int *longind;
+     int long_only;
+{
+  woptarg = NULL;
+
+  if (woptind == 0)
+    optstring = _wgetopt_initialize (optstring);
+
+  if (nextchar == NULL || *nextchar == '\0')
+    {
+      /* Advance to the next ARGV-element.  */
+
+      if (ordering == PERMUTE)
+	{
+	  /* If we have just processed some options following some non-options,
+	     exchange them so that the options come first.  */
+
+	  if (first_nonopt != last_nonopt && last_nonopt != woptind)
+	    exchange ((wchar_t **) argv);
+	  else if (last_nonopt != woptind)
+	    first_nonopt = woptind;
+
+	  /* Skip any additional non-options
+	     and extend the range of non-options previously skipped.  */
+
+	  while (woptind < argc
+		 && (argv[woptind][0] != '-' || argv[woptind][1] == '\0'))
+	    woptind++;
+	  last_nonopt = woptind;
+	}
+
+      /* The special ARGV-element `--' means premature end of options.
+	 Skip it like a null option,
+	 then exchange with previous non-options as if it were an option,
+	 then skip everything else like a non-option.  */
+
+      if (woptind != argc && !wcscmp (argv[woptind], L"--"))
+	{
+	  woptind++;
+
+	  if (first_nonopt != last_nonopt && last_nonopt != woptind)
+	    exchange ((wchar_t **) argv);
+	  else if (first_nonopt == last_nonopt)
+	    first_nonopt = woptind;
+	  last_nonopt = argc;
+
+	  woptind = argc;
+	}
+
+      /* If we have done all the ARGV-elements, stop the scan
+	 and back over any non-options that we skipped and permuted.  */
+
+      if (woptind == argc)
+	{
+	  /* Set the next-arg-index to point at the non-options
+	     that we previously skipped, so the caller will digest them.  */
+	  if (first_nonopt != last_nonopt)
+	    woptind = first_nonopt;
+	  return EOF;
+	}
+
+      /* If we have come to a non-option and did not permute it,
+	 either stop the scan or describe it to the caller and pass it by.  */
+
+      if ((argv[woptind][0] != '-' || argv[woptind][1] == '\0'))
+	{
+	  if (ordering == REQUIRE_ORDER)
+	    return EOF;
+	  woptarg = argv[woptind++];
+	  return 1;
+	}
+
+      /* We have found another option-ARGV-element.
+	 Skip the initial punctuation.  */
+
+      nextchar = (argv[woptind] + 1
+		  + (longopts != NULL && argv[woptind][1] == '-'));
+    }
+
+  /* Decode the current option-ARGV-element.  */
+
+  /* Check whether the ARGV-element is a long option.
+
+     If long_only and the ARGV-element has the form "-f", where f is
+     a valid short option, don't consider it an abbreviated form of
+     a long option that starts with f.  Otherwise there would be no
+     way to give the -f short option.
+
+     On the other hand, if there's a long option "fubar" and
+     the ARGV-element is "-fu", do consider that an abbreviation of
+     the long option, just like "--fu", and not "-f" with arg "u".
+
+     This distinction seems to be the most useful approach.  */
+
+  if (longopts != NULL
+      && (argv[woptind][1] == '-'
+	  || (long_only && (argv[woptind][2] || !my_index (optstring, argv[woptind][1])))))
+    {
+      wchar_t *nameend;
+      const struct woption *p;
+      const struct woption *pfound = NULL;
+      int exact = 0;
+      int ambig = 0;
+      int indfound = 0; /* set to zero by Anton */
+      int option_index;
+
+      for (nameend = nextchar; *nameend && *nameend != '='; nameend++)
+	/* Do nothing.  */ ;
+
+      /* Test all long options for either exact match
+	 or abbreviated matches.  */
+      for (p = longopts, option_index = 0; p->name; p++, option_index++)
+	if (!wcsncmp(p->name, nextchar, nameend - nextchar))
+	  {
+	    if ((unsigned int)(nameend - nextchar) == (unsigned int)wcslen (p->name))
+	      {
+		/* Exact match found.  */
+		pfound = p;
+		indfound = option_index;
+		exact = 1;
+		break;
+	      }
+	    else if (pfound == NULL)
+	      {
+		/* First nonexact match found.  */
+		pfound = p;
+		indfound = option_index;
+	      }
+	    else
+	      /* Second or later nonexact match found.  */
+	      ambig = 1;
+	  }
+
+      if (ambig && !exact)
+	{
+	  if (wopterr)
+	    fwprintf (stderr, L"%ls: option `%ls' is ambiguous\n",
+		     argv[0], argv[woptind]);
+	  nextchar += wcslen (nextchar);
+	  woptind++;
+	  return '?';
+	}
+
+      if (pfound != NULL)
+	{
+	  option_index = indfound;
+	  woptind++;
+	  if (*nameend)
+	    {
+	      /* Don't test has_arg with >, because some C compilers don't
+		 allow it to be used on enums.  */
+	      if (pfound->has_arg)
+		woptarg = nameend + 1;
+	      else
+		{
+		  if (wopterr)
+		    {
+		      if (argv[woptind - 1][1] == '-')
+			/* --option */
+			fwprintf (stderr,
+				 L"%ls: option `--%ls' doesn't allow an argument\n",
+				 argv[0], pfound->name);
+		      else
+			/* +option or -option */
+			fwprintf (stderr,
+			     L"%ls: option `%lc%ls' doesn't allow an argument\n",
+			     argv[0], argv[woptind - 1][0], pfound->name);
+		    }
+		  nextchar += wcslen (nextchar);
+		  return '?';
+		}
+	    }
+	  else if (pfound->has_arg == 1)
+	    {
+	      if (woptind < argc)
+		woptarg = argv[woptind++];
+	      else
+		{
+		  if (wopterr)
+		    fwprintf (stderr, L"%ls: option `%ls' requires an argument\n",
+			     argv[0], argv[woptind - 1]);
+		  nextchar += wcslen (nextchar);
+		  return optstring[0] == ':' ? ':' : '?';
+		}
+	    }
+	  nextchar += wcslen (nextchar);
+	  if (longind != NULL)
+	    *longind = option_index;
+	  if (pfound->flag)
+	    {
+	      *(pfound->flag) = pfound->val;
+	      return 0;
+	    }
+	  return pfound->val;
+	}
+
+      /* Can't find it as a long option.  If this is not getopt_long_only,
+	 or the option starts with '--' or is not a valid short
+	 option, then it's an error.
+	 Otherwise interpret it as a short option.  */
+      if (!long_only || argv[woptind][1] == '-'
+	  || my_index (optstring, *nextchar) == NULL)
+	{
+	  if (wopterr)
+	    {
+	      if (argv[woptind][1] == '-')
+		/* --option */
+		fwprintf (stderr, L"%ls: unrecognized option `--%ls'\n",
+			 argv[0], nextchar);
+	      else
+		/* +option or -option */
+		fwprintf (stderr, L"%ls: unrecognized option `%lc%ls'\n",
+			 argv[0], argv[woptind][0], nextchar);
+	    }
+	  nextchar = (wchar_t *) L"";
+	  woptind++;
+	  return '?';
+	}
+    }
+
+  /* Look at and handle the next short option-character.  */
+
+  {
+    wchar_t c = *nextchar++;
+    wchar_t *temp = my_index (optstring, c);
+
+    /* Increment `woptind' when we start to process its last character.  */
+    if (*nextchar == '\0')
+      ++woptind;
+
+    if (temp == NULL || c == ':')
+      {
+	if (wopterr)
+	  {
+	    if (posixly_correct)
+	      /* 1003.2 specifies the format of this message.  */
+	      fwprintf (stderr, L"%ls: illegal option -- %lc\n", argv[0], c);
+	    else
+	      fwprintf (stderr, L"%ls: invalid option -- %lc\n", argv[0], c);
+	  }
+	woptopt = c;
+	return '?';
+      }
+    if (temp[1] == ':')
+      {
+	if (temp[2] == ':')
+	  {
+	    /* This is an option that accepts an argument optionally.  */
+	    if (*nextchar != '\0')
+	      {
+		woptarg = nextchar;
+		woptind++;
+	      }
+	    else
+	      woptarg = NULL;
+	    nextchar = NULL;
+	  }
+	else
+	  {
+	    /* This is an option that requires an argument.  */
+	    if (*nextchar != '\0')
+	      {
+		woptarg = nextchar;
+		/* If we end this ARGV-element by taking the rest as an arg,
+		   we must advance to the next element now.  */
+		woptind++;
+	      }
+	    else if (woptind == argc)
+	      {
+		if (wopterr)
+		  {
+		    /* 1003.2 specifies the format of this message.  */
+		    fwprintf (stderr, L"%ls: option requires an argument -- %lc\n",
+			     argv[0], c);
+		  }
+		woptopt = c;
+		if (optstring[0] == ':')
+		  c = ':';
+		else
+		  c = '?';
+	      }
+	    else
+	      /* We already incremented `woptind' once;
+		 increment it again when taking next ARGV-elt as argument.  */
+	      woptarg = argv[woptind++];
+	    nextchar = NULL;
+	  }
+      }
+    return c;
+  }
+}
+
+int
+wgetopt (argc, argv, optstring)
+     int argc;
+     wchar_t *const *argv;
+     const wchar_t *optstring;
+{
+  return _wgetopt_internal (argc, argv, optstring,
+			   (const struct woption *) 0,
+			   (int *) 0,
+			   0);
+}
+
+int
+wgetopt_long (argc, argv, options, long_options, opt_index)
+	int argc;
+	wchar_t *const *argv;
+	const wchar_t *options;
+	const struct woption *long_options;	
+	int *opt_index;
+{
+	return _wgetopt_internal (argc, argv, options, long_options, opt_index, 0);
+}
+
+int
+wgetopt_long_only (argc, argv, options, long_options, opt_index)
+	int argc;
+	wchar_t *const *argv;
+	const wchar_t *options;
+	const struct woption *long_options;	
+	int *opt_index;
+{
+	return _wgetopt_internal (argc, argv, options, long_options, opt_index, 1);
+}

wgetopt.h

@@ -0,0 +1,145 @@
+/** \file wgetopt.h
+	The getopt librar for wide character strings. 
+
+	This is simply the gnu getopt library, but converted for use with wchar_t instead of char. This is not usually useful since the argv array is always defined to be of type char**, but in fish, all internal commands use wide characters and hence this library is usefull.
+*/
+
+
+/* Declarations for getopt.
+   Copyright (C) 1989, 90, 91, 92, 93, 94 Free Software Foundation, Inc.
+
+This file is part of the GNU C Library.  Its master source is NOT part of
+the C library, however.  The master source lives in /gd/gnu/lib.
+
+The GNU C Library is free software; you can redistribute it and/or
+modify it under the terms of the GNU Library General Public License as
+published by the Free Software Foundation; either version 2 of the
+License, or (at your option) any later version.
+
+The GNU C Library is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+Library General Public License for more details.
+
+You should have received a copy of the GNU Library General Public
+License along with the GNU C Library; see the file COPYING.LIB.  If
+not, write to the Free Software Foundation, Inc., 675 Mass Ave,
+Cambridge, MA 02139, USA.  */
+
+#ifndef _WGETOPT_H
+#define _WGETOPT_H 1
+
+#ifdef	__cplusplus
+extern "C" {
+#endif
+
+/** For communication from `getopt' to the caller.
+   When `getopt' finds an option that takes an argument,
+   the argument value is returned here.
+   Also, when `ordering' is RETURN_IN_ORDER,
+   each non-option ARGV-element is returned here.  */
+
+extern wchar_t *woptarg;
+
+/** Index in ARGV of the next element to be scanned.
+   This is used for communication to and from the caller
+   and for communication between successive calls to `getopt'.
+
+   On entry to `getopt', zero means this is the first call; initialize.
+
+   When `getopt' returns EOF, this is the index of the first of the
+   non-option elements that the caller should itself scan.
+
+   Otherwise, `optind' communicates from one call to the next
+   how much of ARGV has been scanned so far.  */
+
+extern int woptind;
+
+/** Callers store zero here to inhibit the error message `getopt' prints
+   for unrecognized options.  */
+
+extern int wopterr;
+
+/** Set to an option character which was unrecognized.  */
+
+extern int woptopt;
+
+/** Describe the long-named options requested by the application.
+   The LONG_OPTIONS argument to getopt_long or getopt_long_only is a vector
+   of `struct option' terminated by an element containing a name which is
+   zero.
+
+   The field `has_arg' is:
+   no_argument		(or 0) if the option does not take an argument,
+   required_argument	(or 1) if the option requires an argument,
+   optional_argument 	(or 2) if the option takes an optional argument.
+
+   If the field `flag' is not NULL, it points to a variable that is set
+   to the value given in the field `val' when the option is found, but
+   left unchanged if the option is not found.
+
+   To have a long-named option do something other than set an `int' to
+   a compiled-in constant, such as set a value from `optarg', set the
+   option's `flag' field to zero and its `val' field to a nonzero
+   value (the equivalent single-letter option character, if there is
+   one).  For long options that have a zero `flag' field, `getopt'
+   returns the contents of the `val' field.  */
+
+struct woption
+{
+#if defined (__STDC__) && __STDC__
+  const wchar_t *name;
+#else
+  wchar_t *name;
+#endif
+  /* has_arg can't be an enum because some compilers complain about
+     type mismatches in all the code that assumes it is an int.  */
+  int has_arg;
+  int *flag;
+  int val;
+};
+
+/* Names for the values of the `has_arg' field of `struct option'.  */
+
+#define	no_argument		0
+#define required_argument	1
+#define optional_argument	2
+
+#if defined (__STDC__) && __STDC__
+#ifdef __GNU_LIBRARY__
+/* Get options from argument list */
+extern int wgetopt (int argc, wchar_t *const *argv, const wchar_t *shortopts);
+#else /* not __GNU_LIBRARY__ */
+/* Get options from argument list */
+extern int wgetopt ();
+#endif /* __GNU_LIBRARY__ */
+/* Get options from argument list */
+extern int wgetopt_long (int argc, wchar_t *const *argv, const wchar_t *shortopts,
+		        const struct woption *longopts, int *longind);
+/* Get options from argument list */
+extern int wgetopt_long_only (int argc, wchar_t *const *argv,
+			     const wchar_t *shortopts,
+		             const struct woption *longopts, int *longind);
+
+/** Internal only.  Users should not call this directly.  */
+extern int _wgetopt_internal (int argc, wchar_t *const *argv,
+			     const wchar_t *shortopts,
+		             const struct woption *longopts, int *longind,
+			     int long_only);
+#else /* not __STDC__ */
+/* Get options from argument list */
+extern int wgetopt ();
+/* Get options from argument list */
+extern int wgetopt_long ();
+/* Get options from argument list */
+extern int wgetopt_long_only ();
+
+/* Get options from argument list */
+extern int _wgetopt_internal ();
+#endif /* __STDC__ */
+
+#ifdef	__cplusplus
+}
+#endif
+
+#endif /* _WGETOPT_H */

wildcard.c

@@ -0,0 +1,581 @@
+/** \file wildcard.c
+	My own globbing implementation. Needed to implement this to
+	support tab-expansion of globbed parameters.
+
+*/
+
+#include "config.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <limits.h>
+#include <wchar.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <dirent.h>
+#include <errno.h>
+
+#include "util.h"
+#include "wutil.h"
+#include "complete.h"
+#include "common.h"
+#include "wildcard.h"
+#include "complete.h"
+#include "reader.h"
+#include "expand.h"
+
+int wildcard_has( const wchar_t *str, int internal )
+{
+	wchar_t prev=0;
+	if( internal )
+	{
+		for( ; *str; str++ )
+		{
+			if( ( *str == ANY_CHAR ) || (*str == ANY_STRING) || (*str == ANY_STRING_RECURSIVE) )
+				return 1;
+			prev = *str;
+		}
+	}
+	else
+	{
+		for( ; *str; str++ )
+		{
+			if( ( (*str == L'*' ) || (*str == L'?' ) ) && (prev != L'\\') )
+				return 1;
+			prev = *str;
+		}
+	}
+	
+	return 0;
+}
+
+/**
+   Check whether the string str matches the wildcard string wc.
+  
+   \param str String to be matched.
+   \param wc The wildcard.
+   \param is_first Whether files beginning with dots should not be matched against wildcards.
+   \param wc_unescaped Whether the unescaped special character ANY_CHAR abd ANY_STRING should be used instead of '?' and '*' for wildcard matching
+*/
+
+
+static int wildcard_match2( const wchar_t *str, 
+							const wchar_t *wc, 
+							int is_first )
+{
+	
+	if( *str == 0 && *wc==0 )
+		return 1;
+	
+	if( *wc == ANY_STRING || *wc == ANY_STRING_RECURSIVE)
+	{		
+		/* Ignore hidden file */
+		if( is_first && *str == L'.' )
+			return 0;
+		
+		/* Try all submatches */
+		do
+		{
+			if( wildcard_match2( str, wc+1, 0 ) )
+				return 1;
+		}
+		while( *(str++) != 0 );
+		return 0;
+	}
+	if( *wc == ANY_CHAR )
+		return wildcard_match2( str+1, wc+1, 0 );
+	
+	if( *wc == *str )
+		return wildcard_match2( str+1, wc+1, 0 );
+
+	return 0;
+}
+
+/**
+   Matches the string against the wildcard, and if the wildcard is a
+   possible completion of the string, the remainder of the string is
+   inserted into the array_list_t.
+*/
+static int wildcard_complete_internal( const wchar_t *orig, 
+									   const wchar_t *str, 
+									   const wchar_t *wc, 
+									   int is_first,
+									   const wchar_t *desc,
+									   const wchar_t *(*desc_func)(const wchar_t *),
+									   array_list_t *out )
+{
+	if( *wc == 0 && 
+		( ( *str != L'.') || (!is_first)) )
+	{
+		if( !out )
+			return 1;
+		
+		wchar_t *new;
+			
+		if( wcschr( str, PROG_COMPLETE_SEP ) )
+		{
+			/*
+			  This completion has an embedded description, du not use the generic description
+			*/
+			wchar_t *sep;
+				
+			new = wcsdup( str );
+			sep = wcschr(new, PROG_COMPLETE_SEP );
+			*sep = COMPLETE_SEP;
+		}
+		else if( desc_func )
+		{
+			/*
+			  A descripton generating function is specified, use it
+			*/
+			new = wcsdupcat2( str, COMPLETE_SEP_STR, desc_func( orig ), 0);			
+		}
+		else
+		{
+			/*
+			  Append generic description to item, if the description exists
+			*/
+			if( desc && wcslen(desc)>1 )
+				new = wcsdupcat( str, desc );
+			else
+				new = wcsdup( str );
+		}
+
+		if( new )
+		{
+			al_push( out, new );
+		}
+		return 1;
+	}
+	
+	
+	if( *wc == ANY_STRING )
+	{		
+		int res=0;
+		
+		/* Ignore hidden file */
+		if( is_first && str[0] == L'.' )
+			return 0;
+		
+		/* Try all submatches */
+		do
+		{
+			res |= wildcard_complete_internal( orig, str, wc+1, 0, desc, desc_func, out );
+			if( res && !out )
+				break;
+		}
+		while( *str++ != 0 );
+		return res;
+		
+	}
+	else if( *wc == ANY_CHAR )
+	{
+		return wildcard_complete_internal( orig, str+1, wc+1, 0, desc, desc_func, out );
+	}	
+	else if( *wc == *str )
+	{
+		return wildcard_complete_internal( orig, str+1, wc+1, 0, desc, desc_func, out );
+	}
+	return 0;	
+}
+
+int wildcard_complete( const wchar_t *str,
+					   const wchar_t *wc,
+					   const wchar_t *desc,						
+					   const wchar_t *(*desc_func)(const wchar_t *),
+					   array_list_t *out )
+{
+	return wildcard_complete_internal( str, str, wc, 1, desc, desc_func, out );	
+}
+
+
+int wildcard_match( const wchar_t *str, const wchar_t *wc )
+{
+	return wildcard_match2( str, wc, 1 );	
+}
+
+/**
+   Creates a path from the specified directory and filename. 
+*/
+static wchar_t *make_path( const wchar_t *base_dir, const wchar_t *name )
+{
+	
+	wchar_t *long_name;
+	int base_len = wcslen( base_dir );
+	if( !(long_name= malloc( sizeof(wchar_t)*(base_len+wcslen(name)+1) )))
+	{
+		
+		return 0;
+	}					
+	wcscpy( long_name, base_dir );
+	wcscpy(&long_name[base_len], name );
+	return long_name;
+}
+
+
+void get_desc( wchar_t *fn, string_buffer_t *sb, int is_cmd )
+{
+	const wchar_t *desc;
+	
+	struct stat buf;
+	off_t sz;
+	wchar_t *sz_name[]=
+		{
+			L"kB", L"MB", L"GB", L"TB", L"PB", L"EB", L"ZB", L"YB", 0
+		}
+	;
+	
+	
+	sb_clear( sb );
+	
+	if( wstat( fn, &buf ) )
+	{
+		sz=-1;
+	}
+	else
+	{
+		sz = buf.st_size;
+	}
+						
+	desc = complete_get_desc( fn );
+	
+	if( sz >= 0 && S_ISDIR(buf.st_mode) )
+	{
+		sb_append2( sb, desc, 0 );							
+	}
+	else
+	{							
+		sb_append2( sb, desc, L", ", 0 );
+		if( sz < 0 )
+		{
+			sb_append( sb, L"unknown" );
+		}
+		else if( sz < 1 )
+		{
+			sb_append( sb, L"empty" );
+		}
+		else if( sz < 1024 )
+		{
+			sb_printf( sb, L"%dB", sz );
+		}
+		else
+		{
+			int i;
+			
+			for( i=0; sz_name[i]; i++ )
+			{
+				if( sz < (1024*1024) || !sz_name[i+1] )
+				{
+					int isz = sz/1024;
+					if( isz > 9 )
+						sb_printf( sb, L"%d%ls", isz, sz_name[i] );
+					else
+						sb_printf( sb, L"%.1f%ls", (double)sz/1024, sz_name[i] );
+					
+					break;
+				}
+				sz /= 1024;
+
+			}
+		}		
+	}
+}
+
+static int test_flags( wchar_t *filename,
+					   int flags )
+{
+	if( !(flags & EXECUTABLES_ONLY) && !(flags & DIRECTORIES_ONLY) )
+		return 1;
+	
+	struct stat buf;
+	wstat( filename, &buf );
+	
+	if( S_IFDIR & buf.st_mode )
+		return 1;
+	
+	if( flags & EXECUTABLES_ONLY )
+		return ( waccess( filename, X_OK ) == 0);
+
+	return 0;
+}
+
+
+int wildcard_expand( const wchar_t *wc, 
+					 const wchar_t *base_dir,
+					 int flags,
+					 array_list_t *out )
+{
+//	debug( 3, L"WILDCARD_EXPAND %ls in %ls", wc, base_dir );
+
+	if( flags & ACCEPT_INCOMPLETE )
+	{	
+		/* Avoid excessive number of returned matches for wc ending with a * */
+		int len = wcslen(wc);
+		if( len && (wc[len-1]==ANY_STRING) )
+		{
+			wchar_t * foo = wcsdup( wc );
+			foo[len-1]=0;
+			int res = wildcard_expand( foo, base_dir, flags, out );
+			free( foo );
+			return res;			
+		}
+	}
+	
+	struct dirent *next;
+	wchar_t *wc_end = wcschr(wc,L'/');
+	DIR *dir;
+	
+	int res = 0;
+	int base_len = wcslen( base_dir );
+
+	wchar_t *wc_recursive = wcschr( wc, ANY_STRING_RECURSIVE );
+	int is_recursive = 	is_recursive = ( wc_recursive && (!wc_end || wc_recursive < wc_end));
+
+	const wchar_t *dir_string = base_dir[0]==L'\0'?L".":base_dir;
+
+	string_buffer_t sb_desc;
+	
+	sb_init( &sb_desc );
+//	if( accept_incomplete )
+//		wprintf( L"Glob %ls in '%ls'\n", wc, base_dir );//[0]==L'\0'?L".":base_dir );
+	
+/*
+  Test for recursive match string in current segment
+*/	
+	
+	if( !(dir = wopendir( dir_string )))
+	{
+//		if( errno != EACCES && errno != ENOENT )
+//			wperror( L"opendir" );
+		return 0;
+	}
+
+/*
+  Is this segment of the wildcard the last?
+*/
+	if( wc_end == 0 )
+	{
+		/*
+		  Wildcard segment is the last segment
+
+		  Insert all matching files/directories
+		*/
+		if( wc[0]=='\0' )
+		{
+			/*
+			  The last wildcard segment is empty. Insert everything if completing, the directory itself otherwise.
+			*/
+			if( flags & ACCEPT_INCOMPLETE )
+			{
+				while( (next=readdir(dir))!=0 )
+				{
+					if( next->d_name[0] != '.' )
+					{
+						wchar_t *name = str2wcs(next->d_name);
+						if( name == 0 )
+						{
+/*							closedir( dir );*/
+/*							return -1;							*/
+							continue;
+						}
+						wchar_t *long_name = make_path( base_dir, name );
+						
+						if( long_name == 0 )
+						{
+							wperror( L"malloc" );
+							closedir( dir );
+							free(name);
+							return 0;						
+						}
+						
+						if( test_flags( long_name, flags ) )
+						{
+							get_desc( long_name,
+									  &sb_desc,
+									  flags & EXECUTABLES_ONLY );
+							al_push( out,
+									 wcsdupcat(name, (wchar_t *)sb_desc.buff) );
+						}
+						
+						free(name);
+
+						free( long_name );
+					}					
+				}
+			}
+			else
+			{								
+				res = 1;
+				al_push( out, wcsdup( base_dir ) );
+			}							
+		}
+		else
+		{
+			/*
+			  This is the last wildcard segment, and it is not empty. Match files/directories.
+			*/
+			while( (next=readdir(dir))!=0 )
+			{
+				wchar_t *name = str2wcs(next->d_name);
+				if( name == 0 )
+				{
+					continue;
+				}
+				
+/*				wprintf( L"Filen heter %s\n\n\n", next->d_name );*/
+/*				wprintf( L"Match %ls (%s) against %ls\n\n\n", name, "tjo", wc );*/
+				if( flags & ACCEPT_INCOMPLETE )
+				{
+					/*					wprintf( L"match %ls to %ls\n", name, wc );*/
+					
+					wchar_t *long_name = make_path( base_dir, name );
+					if( long_name == 0 )
+					{
+						wperror( L"malloc" );
+						closedir( dir );
+						free(name);
+						return 0;						
+					}
+					/*
+					  Test for matches before stating file, so as to minimize the number of stat calls
+					*/
+					if( wildcard_complete( name,
+										   wc,
+										   L"",
+										   0,
+										   0 ) )
+					{
+
+						
+						if( test_flags( long_name, flags ) )
+						{
+							get_desc( long_name,
+									  &sb_desc, 
+									  flags & EXECUTABLES_ONLY );
+							
+							wildcard_complete( name,
+											   wc,
+											   (wchar_t *)sb_desc.buff,
+											   0,
+											   out );
+						}
+					}
+					
+					free( long_name );
+					
+				}
+				else
+				{
+					if( wildcard_match2( name, wc, 1 ) )
+					{
+						wchar_t *long_name = make_path( base_dir, name );
+						if( long_name == 0 )
+						{
+							wperror( L"malloc" );
+							closedir( dir );
+							free(name);
+							return 0;						
+						}
+						
+						al_push( out, long_name );
+						res = 1;
+					}
+				}
+				free( name );
+			}
+		}
+	}
+	else
+	{
+		/*
+		  Wilcard segment is not the last segment.
+		  Recursively call wildcard_expand for all matching subdirectories.
+		*/
+		wchar_t *wc_str;
+		wchar_t *new_dir;
+		static size_t ln=1024;
+		char * narrow_dir_string = wcs2str( dir_string );
+		
+		if( narrow_dir_string )
+		{
+			ln = pathconf( narrow_dir_string, _PC_NAME_MAX ); /* Find out how long the filename can be in a worst case scenario */
+			if( ln < 0 )
+				ln = 1024;		
+			free( narrow_dir_string );
+		}
+		new_dir= malloc( sizeof(wchar_t)*(base_len+ln+2)  );
+
+		wc_str = wcsndup(wc, wc_end-wc);
+		if( (!new_dir) || (!wc_str) )
+		{
+			if( new_dir ) 
+				free( new_dir );
+			if( wc_str ) 
+				free( wc_str );
+			wperror( L"malloc" );			
+			closedir( dir );
+			return 0;			
+		}
+		wcscpy( new_dir, base_dir );
+		
+		while( (next=readdir(dir))!=0 )
+		{
+			wchar_t *name = str2wcs(next->d_name);
+			if( name == 0 )
+			{
+				continue;
+			}			
+			
+			if( wildcard_match2( name, wc_str, 1 ) )
+			{
+				int new_len;
+				struct stat buf;			
+				wcscpy(&new_dir[base_len], name );
+				free(name);
+				char *dir_str = wcs2str( new_dir );
+				int stat_res;
+				
+				if( !dir_str )
+				{
+					continue;					
+				}
+				
+				stat_res= stat( dir_str, &buf );
+				free( dir_str );
+				
+				if( stat_res )
+				{
+					continue;
+				}
+
+				if( buf.st_mode & S_IFDIR )
+				{
+					new_len = wcslen( new_dir );
+					new_dir[new_len] = L'/';
+					new_dir[new_len+1] = L'\0';
+					switch( wildcard_expand( wc_end + 1, new_dir, flags, out ) )
+					{
+						case 0:
+							break;
+						case 1:
+							res = 1;
+							break;
+					}
+				}								
+			}
+			else
+			{
+				free(name);
+			}			
+		}
+		free( wc_str );
+		free( new_dir );
+	}
+	closedir( dir );
+
+	sb_destroy( &sb_desc );
+
+	return res;
+}
+

wildcard.h

@@ -0,0 +1,80 @@
+
+/** \file wildcard.h 
+
+    My own globbing implementation. Needed to implement this instead
+    of using libs globbing to support tab-expantion of globbed
+    paramaters.
+
+*/
+
+/*
+    These constants are outside the 31 bit character space of USC4,
+	thogh they may clash with WEOF. I need to use characters outside of
+	the regular character space to represent wildcards and such, 
+	in order to do backslash removal before wildcard matching.
+*/
+
+/** Character representing any character except '/' */
+#define ANY_CHAR 0xfffffffe
+
+/** Character representing any character string not containing '/' (A slash) */
+#define ANY_STRING 0xfffffffd
+
+/** Character representing any character string */
+#define ANY_STRING_RECURSIVE 0xfffffff6
+
+/**
+    Expand the wildcard by matching against the filesystem. 
+
+	New strings are allocated using malloc and should be freed by the caller.
+
+	wildcard_expand works by dividing the wildcard into segments at
+	each directory boundary. Each segment is processed separatly. All
+	except the last segment are handled by matching the wildcard
+	segment against all subdirectories of matching directories, and
+	recursively calling wildcard_expand for matches. On the last
+	segment, matching is made to any file, and all matches are
+	inserted to the list.
+
+	If wildcard_expand encounters any errors (such as insufficient
+	priviliges) during matching, no error messages will be printed and
+	wildcard_expand will continue the matching process.
+
+	\param wc The wildcard string
+	\param base_dir The base directory of the filesystem to perform the match against
+	\param status flags for the search. Can be any combination of ACCEPT_INCOMPLETE and EXECUTABLES_ONLY
+	\param out The list in which to put the output
+
+	\return 1 if matches where found, 0 otherwise.
+   
+*/
+int wildcard_expand( const wchar_t *wc, 
+					 const wchar_t *base_dir, 
+					 int flags, 
+					 array_list_t *out );
+/**
+   Test whether the given wildcard matches the string
+
+   \param str The string to test
+   \param wc The wildcard to test against
+   \param wc_unescaped if wc_unescaped is true, \c wildcard_match uses the ANY_CHAR and ANY_STRING characters for globbing, otherwise, the '?' and '*' characters are used
+   \return true if the wildcard matched
+*/
+int wildcard_match( const wchar_t *str, 
+					const wchar_t *wc );
+
+
+/**
+   Check if the specified string contains wildcards
+*/
+int wildcard_has( const wchar_t *str, int internal );
+
+/**
+   Test wildcard completion
+*/
+int wildcard_complete( const wchar_t *str,
+						const wchar_t *wc,
+						const wchar_t *desc,						
+						const wchar_t *(*desc_func)(const wchar_t *),
+						array_list_t *out );
+

wutil.c

@@ -0,0 +1,546 @@
+/** \file wutil.c
+	Wide character equivalents of various standard unix functions. 
+*/
+#include "config.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <wchar.h>
+#include <string.h>
+#include <dirent.h>
+#include <stdarg.h>
+#include <limits.h>
+
+#include "util.h"
+#include "common.h"
+#include "wutil.h"
+
+static char *tmp=0;
+static size_t tmp_len=0;
+
+int c = 0;
+
+void wutil_destroy()
+{
+	free( tmp );
+	tmp=0;
+	tmp_len=0;
+	debug( 3, L"wutil functions called %d times", c );
+}
+
+static char *wutil_wcs2str( const wchar_t *in )
+{
+	c++;
+	
+	size_t new_sz =MAX_UTF8_BYTES*wcslen(in)+1;
+	if( tmp_len < new_sz )
+	{
+		free( tmp );
+		tmp = malloc( new_sz );
+		if( !tmp )
+		{
+			die_mem();
+		}
+		tmp_len = new_sz;
+	}
+	
+	wcstombs( tmp, in, tmp_len );
+	return tmp;
+}
+
+wchar_t *wgetcwd( wchar_t *buff, size_t sz )
+{
+	char buffc[sz*MAX_UTF8_BYTES];
+	char *res = getcwd( buffc, sz*MAX_UTF8_BYTES );
+	if( !res )
+		return 0;
+	
+	if( (size_t)-1 == mbstowcs( buff, buffc, sizeof( wchar_t ) * sz ) )
+	{
+		return 0;		
+	}	
+	return buff;
+}
+
+int wchdir( const wchar_t * dir )
+{
+	char *tmp = wutil_wcs2str(dir);
+	return chdir( tmp );
+}
+
+FILE *wfopen(const wchar_t *path, const char *mode)
+{
+	
+	char *tmp =wutil_wcs2str(path);
+	FILE *res=0;
+	if( tmp )
+	{
+		res = fopen(tmp, mode);
+		
+	}
+	return res;	
+}
+
+FILE *wfreopen(const wchar_t *path, const char *mode, FILE *stream)
+{
+	char *tmp =wutil_wcs2str(path);
+	FILE *res=0;
+	if( tmp )
+	{
+		res = freopen(tmp, mode, stream);
+	}
+	return res;	
+}
+
+
+
+int wopen(const wchar_t *pathname, int flags, ...)
+{
+	char *tmp =wutil_wcs2str(pathname);
+	int res=-1;
+	va_list argp;
+	
+	if( tmp )
+	{
+		va_start( argp, flags );
+		
+		if( ! (flags & O_CREAT) )
+			res = open(tmp, flags);
+		else
+			res = open(tmp, flags, va_arg(argp, int) );
+		
+		va_end( argp );
+	}
+	
+    return res;
+}
+
+int wcreat(const wchar_t *pathname, mode_t mode)
+{
+    char *tmp =wutil_wcs2str(pathname);
+    int res = -1;
+	if( tmp )
+	{
+		res= creat(tmp, mode);
+	}
+	
+    return res;
+}
+
+DIR *wopendir(const wchar_t *name)
+{
+	char *tmp =wutil_wcs2str(name);
+	DIR *res = 0;
+	if( tmp ) 
+	{
+		res = opendir(tmp);
+	}
+	
+	return res;
+}
+
+int wstat(const wchar_t *file_name, struct stat *buf)
+{
+	char *tmp =wutil_wcs2str(file_name);
+	int res = -1;
+
+	if( tmp )
+	{
+		res = stat(tmp, buf);
+	}
+	
+	return res;	
+}
+
+int lwstat(const wchar_t *file_name, struct stat *buf)
+{
+	char *tmp =wutil_wcs2str(file_name);
+	int res = -1;
+
+	if( tmp )
+	{
+		res = lstat(tmp, buf);
+	}
+	
+	return res;	
+}
+
+
+int waccess(const wchar_t *file_name, int mode)
+{
+	char *tmp =wutil_wcs2str(file_name);
+	int res = -1;
+	if( tmp )
+	{
+		res= access(tmp, mode);
+	}	
+	return res;	
+}
+
+void wperror(const wchar_t *s)
+{
+	if( s != 0 )
+	{
+		fwprintf( stderr, L"%ls: ", s );
+	}
+	fwprintf( stderr, L"%s\n", strerror( errno ) );
+}
+
+
+#if !HAVE_WPRINTF
+/*
+  Here is my own implementation of *wprintf, included since NetBSD does
+  not provide one of it's own.
+*/
+
+/**
+   This function is defined to help vgwprintf when it wants to call
+   itself recursively
+*/
+static int gwprintf( void (*writer)(wchar_t), 
+					 const wchar_t *filter, 
+					 ... );
+
+
+/**
+   Generic formatting function. All other formatting functions are
+   secretly a wrapper around this function.
+*/
+static int vgwprintf( void (*writer)(wchar_t), 
+					  const wchar_t *filter, 
+					  va_list va )
+{
+	const wchar_t *filter_org=filter;
+	int count=0;
+	
+	for( ;*filter; filter++)
+	{
+		if(*filter == L'%')
+		{
+			int i;
+			int is_long=0;
+			int width = 0;
+			filter++;
+			int loop=1;
+			int precision=INT_MAX;
+			
+			while( loop )
+			{
+				switch(*filter)
+				{
+					case L'l':
+						/* Long variable */
+						is_long++;
+						filter++;
+						break;
+					case L'*':
+						/* Set minimum field width */
+						width = va_arg( va, int );
+						filter++;
+						break;
+					case L'.':
+						/* 
+						   Set precision.
+						   Hasn't been tested enough yet, so I don't really trust it.
+						*/
+						filter++;
+						if( *filter == L'*' )
+						{
+							precision = va_arg( va, int );
+						}
+						else
+						{
+							while( (*filter >= L'0') && (*filter <= L'9'))
+							{
+								precision=10*precision+(*filter - L'0');
+							}							
+						}
+						break;
+						
+					default:
+						loop=0;
+						break;
+				}
+			}
+			switch( *filter )
+			{
+				case L'c':
+				{
+					wchar_t c;
+					
+					c = is_long?va_arg(va, wchar_t):btowc(va_arg(va, int));
+					if( width )
+					{
+						int i;
+						for( i=1; i<width; i++ )
+						{
+							writer( L' ' );
+							count++;
+						}
+					}
+					if( precision != 0 )
+						writer( c );
+					count++;
+					
+					break;
+				}
+				case L's':
+				{
+					wchar_t *ss = is_long?va_arg(va, wchar_t*):str2wcs(va_arg(va, char*));
+					
+					if( !ss )
+						return -1;
+					
+					if( width )
+					{
+						int i;
+						for( i=wcslen(ss); i<width; i++ )
+						{
+							writer( L' ' );
+							count++;
+						}
+					}
+
+					wchar_t *s=ss;
+					int precount = count;
+					
+					while( *s )
+					{
+						if( (precision <= (count-precount) ) )
+							break;
+
+						writer( *(s++) );
+						count++;
+					}
+					
+					if( !is_long )
+						free( ss );
+					
+					break;
+				}
+
+				case L'd':
+				case L'i':
+				{
+					char str[32];
+
+					switch( is_long )
+					{
+						case 0:
+						{
+							int d = va_arg( va, int );
+							snprintf( str, 32, "%.*d", precision, d );
+							break;
+						}
+						
+						case 1:
+						{
+							long d = va_arg( va, long );
+							snprintf( str, 32, "%.*ld", precision, d );
+							break;
+						}
+						
+						case 2:
+						{
+							long long d = va_arg( va, long long );
+							snprintf( str, 32, "%.*lld", precision, d );
+							break;
+						}
+						
+						default:
+							return -1;
+					}
+					
+					if( width )
+					{
+						int i;
+						for( i=strlen(str); i<width; i++ )
+						{
+							writer( L' ' );
+							count++;
+						}
+					}
+										
+					int c = gwprintf( writer, L"%s", str );
+					if( c==-1 )
+						return -1;
+					else
+						count += c;
+					
+					break;
+				}
+
+				case L'u':
+				{
+					char str[32];
+					
+
+					switch( is_long )
+					{
+						case 0:
+						{
+							unsigned d = va_arg( va, unsigned );
+							snprintf( str, 32, "%d", d );
+							break;
+						}
+						
+						case 1:
+						{
+							unsigned long d = va_arg( va, unsigned long );
+							snprintf( str, 32, "%ld", d );
+							break;
+						}
+						
+						case 2:
+						{
+							unsigned long long d = va_arg( va, unsigned long long );
+							snprintf( str, 32, "%lld", d );
+							break;
+						}
+						
+						default:
+							return -1;
+					}
+					
+					if( width )
+					{
+						int i;
+						for( i=strlen(str); i<width; i++ )
+						{
+							writer( L' ' );
+							count++;
+						}
+					}
+
+					int c = gwprintf( writer, L"%s", str );
+					if( c==-1 )
+						return -1;
+					else
+						count += c;
+					
+					break;
+				}
+
+				case L'n':
+				{
+					int *n = va_arg( va, int *);
+					
+					*n = count;					
+					break;
+				}
+				default:
+					debug( 0, L"Unknown switch %lc in string %ls\n", *filter, filter_org );
+					exit(1);
+			}
+		}
+		else
+		{
+			writer( *filter );
+			count++;
+		}
+	}
+
+	return count;
+}
+
+static int gwprintf( void (*writer)(wchar_t), 
+					 const wchar_t *filter, 
+					 ... )
+{
+	va_list va;
+	va_start( va, filter );
+	int written=vgwprintf( writer,
+						   filter,
+						   va );
+	va_end( va );
+	return written;
+}
+
+/**
+   Holds data for swprintf writer
+*/
+static struct
+{
+	int count;
+	int max;
+	wchar_t *pos;
+}
+sw_data;
+
+/**
+   Writer for swprintf
+*/
+static void sw_writer( wchar_t c )
+{
+	if( sw_data.count < sw_data.max )
+		*(sw_data.pos++)=c;	
+	sw_data.count++;
+}
+
+
+int swprintf( wchar_t *out, size_t n, const wchar_t *filter, ... )
+{
+	va_list va;
+	va_start( va, filter );
+	sw_data.pos=out;
+	sw_data.max=n;
+	sw_data.count=0;
+	int written=vgwprintf( &sw_writer,
+						   filter,
+						   va );
+	if( written < n )
+	{
+		*sw_data.pos = 0;
+	}
+	else
+	{
+		written=-1;
+	}
+	
+	va_end( va );
+	return written;
+}
+
+/**
+   Holds auxiliary data for fwprintf and wprintf writer
+*/
+static FILE *fw_data;
+
+static void fw_writer( wchar_t c )
+{
+	putw( c, fw_data );
+}
+
+/**
+   Writer for fwprintf and wprintf
+*/
+int fwprintf( FILE *f, const wchar_t *filter, ... )
+{
+	va_list va;
+	va_start( va, filter );
+	fw_data = f;
+	
+	int written=vgwprintf( &fw_writer, filter, va );
+	va_end( va );
+	return written;
+}
+
+int wprintf( const wchar_t *filter, ... )
+{
+	va_list va;
+	va_start( va, filter );
+	fw_data = stdout;
+	
+	int written=vgwprintf( &fw_writer, filter, va );
+	va_end( va );
+	return written;
+}
+
+#endif
+

wutil.h

@@ -0,0 +1,111 @@
+/** \file wutil.h
+
+  Prototypes for wide character equivalents of various standard unix
+  functions. 
+
+*/
+#ifndef WUTIL_HH
+#define WUTIL_HH
+
+#include <sys/types.h>
+#include <dirent.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+
+/**
+   Call this function on exit to free internal wutil resources
+*/
+void wutil_destroy();
+
+/**
+   Wide character version of fopen().
+*/
+FILE *wfopen(const wchar_t *path, const char *mode);
+
+/**
+   Wide character version of freopen().
+*/
+FILE *wfreopen(const wchar_t *path, const char *mode, FILE *stream);
+
+/**
+   Wide character version of open().
+*/
+int wopen(const wchar_t *pathname, int flags, ...);
+
+/**
+   Wide character version of creat().
+*/
+int wcreat(const wchar_t *pathname, mode_t mode);
+
+
+/**
+   Wide character version of opendir().
+*/
+DIR *wopendir(const wchar_t *name);
+
+/**
+   Wide character version of stat().
+*/
+int wstat(const wchar_t *file_name, struct stat *buf);
+
+/**
+   Wide character version of lstat().
+*/
+int lwstat(const wchar_t *file_name, struct stat *buf);
+
+/**
+   Wide character version of access().
+*/
+int waccess(const wchar_t *pathname, int mode);
+
+/**
+   Wide character version of perror().
+*/
+void wperror(const wchar_t *s);
+
+/**
+   Wide character version of getcwd().
+*/
+wchar_t *wgetcwd( wchar_t *buff, size_t sz );
+
+/**
+   Wide character version of chdir()
+*/
+int wchdir( const wchar_t * dir );
+
+
+#if !HAVE_WPRINTF
+
+/**
+   Print formated string. Some operating systems (Like NetBSD) do not
+   have wide string formating functions.  Therefore we implement our
+   own. Not at all complete. Supports wide and narrow characters,
+   strings and decimal numbers, position (%n), field width and
+   precision.
+*/
+int fwprintf( FILE *f, const wchar_t *format, ... );
+
+
+/**
+   Print formated string. Some operating systems (Like NetBSD) do not
+   have wide string formating functions.  Therefore we define our
+   own. Not at all complete. Supports wide and narrow characters,
+   strings and decimal numbers, position (%n), field width and
+   precision.
+*/
+int swprintf( wchar_t *str, size_t l, const wchar_t *format, ... );
+
+/**
+   Print formated string. Some operating systems (Like NetBSD) do not
+   have wide string formating functions.  Therefore we define our
+   own. Not at all complete. Supports wide and narrow characters,
+   strings and decimal numbers, position (%n), field width and
+   precision.
+*/
+int wprintf( const wchar_t *format, ... );
+
+
+#endif
+
+#endif

xdgmime.c

@@ -0,0 +1,715 @@
+/* -*- mode: C; c-file-style: "gnu" -*- */
+/* xdgmime.c: XDG Mime Spec mime resolver.  Based on version 0.11 of the spec.
+ *
+ * More info can be found at http://www.freedesktop.org/standards/
+ * 
+ * Copyright (C) 2003,2004  Red Hat, Inc.
+ * Copyright (C) 2003,2004  Jonathan Blandford <jrb@alum.mit.edu>
+ *
+ * Licensed under the Academic Free License version 2.0
+ * Or under the following terms:
+ * 
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 02111-1307, USA.
+ */
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif
+
+#include "xdgmime.h"
+#include "xdgmimeint.h"
+#include "xdgmimeglob.h"
+#include "xdgmimemagic.h"
+#include "xdgmimealias.h"
+#include "xdgmimeparent.h"
+#include <stdio.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/time.h>
+#include <unistd.h>
+#include <assert.h>
+
+typedef struct XdgDirTimeList XdgDirTimeList;
+typedef struct XdgCallbackList XdgCallbackList;
+
+static int need_reread = TRUE;
+static time_t last_stat_time = 0;
+
+static XdgGlobHash *global_hash = NULL;
+static XdgMimeMagic *global_magic = NULL;
+static XdgAliasList *alias_list = NULL;
+static XdgParentList *parent_list = NULL;
+static XdgDirTimeList *dir_time_list = NULL;
+static XdgCallbackList *callback_list = NULL;
+const char *xdg_mime_type_unknown = "application/octet-stream";
+
+
+enum
+  {
+	XDG_CHECKED_UNCHECKED,
+	XDG_CHECKED_VALID,
+	XDG_CHECKED_INVALID
+  };
+
+struct XdgDirTimeList
+{
+  time_t mtime;
+  char *directory_name;
+  int checked;
+  XdgDirTimeList *next;
+};
+
+struct XdgCallbackList
+{
+  XdgCallbackList *next;
+  XdgCallbackList *prev;
+  int              callback_id;
+  XdgMimeCallback  callback;
+  void            *data;
+  XdgMimeDestroy   destroy;
+};
+
+/* Function called by xdg_run_command_on_dirs.  If it returns TRUE, further
+ * directories aren't looked at */
+typedef int (*XdgDirectoryFunc) (const char *directory,
+								 void       *user_data);
+
+static XdgDirTimeList *
+xdg_dir_time_list_new (void)
+{
+  XdgDirTimeList *retval;
+
+  retval = calloc (1, sizeof (XdgDirTimeList));
+  retval->checked = XDG_CHECKED_UNCHECKED;
+
+  return retval;
+}
+
+static void
+xdg_dir_time_list_free (XdgDirTimeList *list)
+{
+  XdgDirTimeList *next;
+
+  while (list)
+    {
+      next = list->next;
+      free (list->directory_name);
+      free (list);
+      list = next;
+    }
+}
+
+static int
+xdg_mime_init_from_directory (const char *directory)
+{
+  char *file_name;
+  struct stat st;
+  XdgDirTimeList *list;
+
+  assert (directory != NULL);
+
+  file_name = malloc (strlen (directory) + strlen ("/mime/globs") + 1);
+  strcpy (file_name, directory); strcat (file_name, "/mime/globs");
+  if (stat (file_name, &st) == 0)
+    {
+      _xdg_mime_glob_read_from_file (global_hash, file_name);
+
+      list = xdg_dir_time_list_new ();
+      list->directory_name = file_name;
+      list->mtime = st.st_mtime;
+      list->next = dir_time_list;
+      dir_time_list = list;
+    }
+  else
+    {
+      free (file_name);
+    }
+
+  file_name = malloc (strlen (directory) + strlen ("/mime/magic") + 1);
+  strcpy (file_name, directory); strcat (file_name, "/mime/magic");
+  if (stat (file_name, &st) == 0)
+    {
+      _xdg_mime_magic_read_from_file (global_magic, file_name);
+
+      list = xdg_dir_time_list_new ();
+      list->directory_name = file_name;
+      list->mtime = st.st_mtime;
+      list->next = dir_time_list;
+      dir_time_list = list;
+    }
+  else
+    {
+      free (file_name);
+    }
+
+  file_name = malloc (strlen (directory) + strlen ("/mime/aliases") + 1);
+  strcpy (file_name, directory); strcat (file_name, "/mime/aliases");
+  _xdg_mime_alias_read_from_file (alias_list, file_name);
+  free (file_name);
+
+  file_name = malloc (strlen (directory) + strlen ("/mime/subclasses") + 1);
+  strcpy (file_name, directory); strcat (file_name, "/mime/subclasses");
+  _xdg_mime_parent_read_from_file (parent_list, file_name);
+  free (file_name);
+
+  return FALSE; /* Keep processing */
+}
+
+/* Runs a command on all the directories in the search path */
+static void
+xdg_run_command_on_dirs (XdgDirectoryFunc  func,
+						 void             *user_data)
+{
+  const char *xdg_data_home;
+  const char *xdg_data_dirs;
+  const char *ptr;
+
+  xdg_data_home = getenv ("XDG_DATA_HOME");
+  if (xdg_data_home)
+    {
+      if ((func) (xdg_data_home, user_data))
+		return;
+    }
+  else
+    {
+      const char *home;
+
+      home = getenv ("HOME");
+      if (home != NULL)
+		{
+		  char *guessed_xdg_home;
+		  int stop_processing;
+
+		  guessed_xdg_home = malloc (strlen (home) + strlen ("/.local/share/") + 1);
+		  strcpy (guessed_xdg_home, home);
+		  strcat (guessed_xdg_home, "/.local/share/");
+		  stop_processing = (func) (guessed_xdg_home, user_data);
+		  free (guessed_xdg_home);
+
+		  if (stop_processing)
+			return;
+		}
+    }
+
+  xdg_data_dirs = getenv ("XDG_DATA_DIRS");
+  if (xdg_data_dirs == NULL)
+    xdg_data_dirs = "/usr/local/share/:/usr/share/";
+
+  ptr = xdg_data_dirs;
+
+  while (*ptr != '\000')
+    {
+      const char *end_ptr;
+      char *dir;
+      int len;
+      int stop_processing;
+
+      end_ptr = ptr;
+      while (*end_ptr != ':' && *end_ptr != '\000')
+		end_ptr ++;
+
+      if (end_ptr == ptr)
+		{
+		  ptr++;
+		  continue;
+		}
+
+      if (*end_ptr == ':')
+		len = end_ptr - ptr;
+      else
+		len = end_ptr - ptr + 1;
+      dir = malloc (len + 1);
+      strncpy (dir, ptr, len);
+      dir[len] = '\0';
+      stop_processing = (func) (dir, user_data);
+      free (dir);
+
+      if (stop_processing)
+		return;
+
+      ptr = end_ptr;
+    }
+}
+
+/* Checks file_path to make sure it has the same mtime as last time it was
+ * checked.  If it has a different mtime, or if the file doesn't exist, it
+ * returns FALSE.
+ *
+ * FIXME: This doesn't protect against permission changes.
+ */
+static int
+xdg_check_file (const char *file_path)
+{
+  struct stat st;
+
+  /* If the file exists */
+  if (stat (file_path, &st) == 0)
+    {
+      XdgDirTimeList *list;
+
+      for (list = dir_time_list; list; list = list->next)
+		{
+		  if (! strcmp (list->directory_name, file_path) &&
+			  st.st_mtime == list->mtime)
+			{
+			  if (list->checked == XDG_CHECKED_UNCHECKED)
+				list->checked = XDG_CHECKED_VALID;
+			  else if (list->checked == XDG_CHECKED_VALID)
+				list->checked = XDG_CHECKED_INVALID;
+
+			  return (list->checked != XDG_CHECKED_VALID);
+			}
+		}
+      return TRUE;
+    }
+
+  return FALSE;
+}
+
+static int
+xdg_check_dir (const char *directory,
+			   int        *invalid_dir_list)
+{
+  int invalid;
+  char *file_name;
+
+  assert (directory != NULL);
+
+  /* Check the globs file */
+  file_name = malloc (strlen (directory) + strlen ("/mime/globs") + 1);
+  strcpy (file_name, directory); strcat (file_name, "/mime/globs");
+  invalid = xdg_check_file (file_name);
+  free (file_name);
+  if (invalid)
+    {
+      *invalid_dir_list = TRUE;
+      return TRUE;
+    }
+
+  /* Check the magic file */
+  file_name = malloc (strlen (directory) + strlen ("/mime/magic") + 1);
+  strcpy (file_name, directory); strcat (file_name, "/mime/magic");
+  invalid = xdg_check_file (file_name);
+  free (file_name);
+  if (invalid)
+    {
+      *invalid_dir_list = TRUE;
+      return TRUE;
+    }
+
+  return FALSE; /* Keep processing */
+}
+
+/* Walks through all the mime files stat()ing them to see if they've changed.
+ * Returns TRUE if they have. */
+static int
+xdg_check_dirs (void)
+{
+  XdgDirTimeList *list;
+  int invalid_dir_list = FALSE;
+
+  for (list = dir_time_list; list; list = list->next)
+    list->checked = XDG_CHECKED_UNCHECKED;
+
+  xdg_run_command_on_dirs ((XdgDirectoryFunc) xdg_check_dir,
+						   &invalid_dir_list);
+
+  if (invalid_dir_list)
+    return TRUE;
+
+  for (list = dir_time_list; list; list = list->next)
+    {
+      if (list->checked != XDG_CHECKED_VALID)
+		return TRUE;
+    }
+
+  return FALSE;
+}
+
+/* We want to avoid stat()ing on every single mime call, so we only look for
+ * newer files every 5 seconds.  This will return TRUE if we need to reread the
+ * mime data from disk.
+ */
+static int
+xdg_check_time_and_dirs (void)
+{
+  struct timeval tv;
+  time_t current_time;
+  int retval = FALSE;
+
+  gettimeofday (&tv, NULL);
+  current_time = tv.tv_sec;
+
+  if (current_time >= last_stat_time + 5)
+    {
+      retval = xdg_check_dirs ();
+      last_stat_time = current_time;
+    }
+
+  return retval;
+}
+
+/* Called in every public function.  It reloads the hash function if need be.
+ */
+static void
+xdg_mime_init (void)
+{
+  if (xdg_check_time_and_dirs ())
+    {
+      xdg_mime_shutdown ();
+    }
+
+  if (need_reread)
+    {
+      global_hash = _xdg_glob_hash_new ();
+      global_magic = _xdg_mime_magic_new ();
+      alias_list = _xdg_mime_alias_list_new ();
+      parent_list = _xdg_mime_parent_list_new ();
+
+      xdg_run_command_on_dirs ((XdgDirectoryFunc) xdg_mime_init_from_directory,
+							   NULL);
+
+      need_reread = FALSE;
+    }
+}
+
+const char *
+xdg_mime_get_mime_type_for_data (const void *data,
+								 size_t      len)
+{
+  const char *mime_type;
+
+  xdg_mime_init ();
+
+  mime_type = _xdg_mime_magic_lookup_data (global_magic, data, len);
+
+  if (mime_type)
+    return mime_type;
+
+  return XDG_MIME_TYPE_UNKNOWN;
+}
+
+const char *
+xdg_mime_get_mime_type_for_file (const char *file_name)
+{
+  const char *mime_type;
+  FILE *file;
+  unsigned char *data;
+  int max_extent;
+  int bytes_read;
+  struct stat statbuf;
+  const char *base_name;
+
+  if (file_name == NULL)
+    return NULL;
+  if (! _xdg_utf8_validate (file_name))
+    return NULL;
+
+  xdg_mime_init ();
+
+  base_name = _xdg_get_base_name (file_name);
+  mime_type = xdg_mime_get_mime_type_from_file_name (base_name);
+
+  if (mime_type != XDG_MIME_TYPE_UNKNOWN)
+    return mime_type;
+
+  if (stat (file_name, &statbuf) != 0)
+    return XDG_MIME_TYPE_UNKNOWN;
+
+  if (!S_ISREG (statbuf.st_mode))
+    return XDG_MIME_TYPE_UNKNOWN;
+
+  /* FIXME: Need to make sure that max_extent isn't totally broken.  This could
+   * be large and need getting from a stream instead of just reading it all
+   * in. */
+  max_extent = _xdg_mime_magic_get_buffer_extents (global_magic);
+  data = malloc (max_extent);
+  if (data == NULL)
+    return XDG_MIME_TYPE_UNKNOWN;
+        
+  file = fopen (file_name, "r");
+  if (file == NULL)
+    {
+      free (data);
+      return XDG_MIME_TYPE_UNKNOWN;
+    }
+
+  bytes_read = fread (data, 1, max_extent, file);
+  if (ferror (file))
+    {
+      free (data);
+      fclose (file);
+      return XDG_MIME_TYPE_UNKNOWN;
+    }
+
+  mime_type = _xdg_mime_magic_lookup_data (global_magic, data, bytes_read);
+
+  free (data);
+  fclose (file);
+
+  if (mime_type)
+    return mime_type;
+
+  return XDG_MIME_TYPE_UNKNOWN;
+}
+
+const char *
+xdg_mime_get_mime_type_from_file_name (const char *file_name)
+{
+  const char *mime_type;
+
+  xdg_mime_init ();
+
+  mime_type = _xdg_glob_hash_lookup_file_name (global_hash, file_name);
+  if (mime_type)
+    return mime_type;
+  else
+    return XDG_MIME_TYPE_UNKNOWN;
+}
+
+int
+xdg_mime_is_valid_mime_type (const char *mime_type)
+{
+  /* FIXME: We should make this a better test
+   */
+  return _xdg_utf8_validate (mime_type);
+}
+
+void
+xdg_mime_shutdown (void)
+{
+  XdgCallbackList *list;
+
+  /* FIXME: Need to make this (and the whole library) thread safe */
+  if (dir_time_list)
+    {
+      xdg_dir_time_list_free (dir_time_list);
+      dir_time_list = NULL;
+    }
+	
+  if (global_hash)
+    {
+      _xdg_glob_hash_free (global_hash);
+      global_hash = NULL;
+    }
+  if (global_magic)
+    {
+      _xdg_mime_magic_free (global_magic);
+      global_magic = NULL;
+    }
+
+  if (alias_list)
+    {
+      _xdg_mime_alias_list_free (alias_list);
+      alias_list = NULL;
+    }
+
+  if( parent_list )
+	{
+	  _xdg_mime_parent_list_free ( parent_list);
+	}
+  
+  
+  for (list = callback_list; list; list = list->next)
+    (list->callback) (list->data);
+
+  need_reread = TRUE;
+}
+
+int
+xdg_mime_get_max_buffer_extents (void)
+{
+  xdg_mime_init ();
+  
+  return _xdg_mime_magic_get_buffer_extents (global_magic);
+}
+
+const char *
+xdg_mime_unalias_mime_type (const char *mime_type)
+{
+  const char *lookup;
+
+  xdg_mime_init ();
+
+  if ((lookup = _xdg_mime_alias_list_lookup (alias_list, mime_type)) != NULL)
+    return lookup;
+
+  return mime_type;
+}
+
+int
+xdg_mime_mime_type_equal (const char *mime_a,
+						  const char *mime_b)
+{
+  const char *unalias_a, *unalias_b;
+
+  xdg_mime_init ();
+
+  unalias_a = xdg_mime_unalias_mime_type (mime_a);
+  unalias_b = xdg_mime_unalias_mime_type (mime_b);
+
+  if (strcmp (unalias_a, unalias_b) == 0)
+    return 1;
+
+  return 0;
+}
+
+int
+xdg_mime_media_type_equal (const char *mime_a,
+						   const char *mime_b)
+{
+  char *sep;
+
+  xdg_mime_init ();
+
+  sep = strchr (mime_a, '/');
+  
+  if (sep && strncmp (mime_a, mime_b, sep - mime_a + 1) == 0)
+    return 1;
+
+  return 0;
+}
+
+#if 0
+static int
+xdg_mime_is_super_type (const char *mime)
+{
+  int length;
+  const char *type;
+
+  length = strlen (mime);
+  type = &(mime[length - 2]);
+
+  if (strcmp (type, "/*") == 0)
+    return 1;
+
+  return 0;
+}
+#endif
+
+int
+xdg_mime_mime_type_subclass (const char *mime,
+							 const char *base)
+{
+  const char *umime, *ubase;
+  const char **parents;
+
+  xdg_mime_init ();
+
+  umime = xdg_mime_unalias_mime_type (mime);
+  ubase = xdg_mime_unalias_mime_type (base);
+
+  if (strcmp (umime, ubase) == 0)
+    return 1;
+
+#if 0  
+  /* Handle supertypes */
+  if (xdg_mime_is_super_type (ubase) &&
+      xdg_mime_media_type_equal (umime, ubase))
+    return 1;
+#endif
+
+  /*  Handle special cases text/plain and application/octet-stream */
+  if (strcmp (ubase, "text/plain") == 0 && 
+      strncmp (umime, "text/", 5) == 0)
+    return 1;
+
+  if (strcmp (ubase, "application/octet-stream") == 0)
+    return 1;
+  
+  parents = _xdg_mime_parent_list_lookup (parent_list, umime);
+  for (; parents && *parents; parents++)
+    {
+      if (xdg_mime_mime_type_subclass (*parents, ubase))
+		return 1;
+    }
+
+  return 0;
+}
+
+const char **
+xdg_mime_get_mime_parents (const char *mime)
+{
+  const char *umime;
+
+  xdg_mime_init ();
+
+  umime = xdg_mime_unalias_mime_type (mime);
+
+  return _xdg_mime_parent_list_lookup (parent_list, umime);
+}
+
+void 
+xdg_mime_dump (void)
+{
+  printf ("*** ALIASES ***\n\n");
+  _xdg_mime_alias_list_dump (alias_list);
+  printf ("\n*** PARENTS ***\n\n");
+  _xdg_mime_parent_list_dump (parent_list);
+}
+
+
+/* Registers a function to be called every time the mime database reloads its files
+ */
+int
+xdg_mime_register_reload_callback (XdgMimeCallback  callback,
+								   void            *data,
+								   XdgMimeDestroy   destroy)
+{
+  XdgCallbackList *list_el;
+  static int callback_id = 1;
+
+  /* Make a new list element */
+  list_el = calloc (1, sizeof (XdgCallbackList));
+  list_el->callback_id = callback_id;
+  list_el->callback = callback;
+  list_el->data = data;
+  list_el->destroy = destroy;
+  list_el->next = callback_list;
+  if (list_el->next)
+    list_el->next->prev = list_el;
+
+  callback_list = list_el;
+  callback_id ++;
+
+  return callback_id - 1;
+}
+
+void
+xdg_mime_remove_callback (int callback_id)
+{
+  XdgCallbackList *list;
+
+  for (list = callback_list; list; list = list->next)
+    {
+      if (list->callback_id == callback_id)
+		{
+		  if (list->next)
+			list->next = list->prev;
+
+		  if (list->prev)
+			list->prev->next = list->next;
+		  else
+			callback_list = list->next;
+
+		  /* invoke the destroy handler */
+		  (list->destroy) (list->data);
+		  free (list);
+		  return;
+		}
+    }
+}

xdgmime.h

@@ -0,0 +1,93 @@
+/* -*- mode: C; c-file-style: "gnu" -*- */
+/* xdgmime.h: XDG Mime Spec mime resolver.  Based on version 0.11 of the spec.
+ *
+ * More info can be found at http://www.freedesktop.org/standards/
+ * 
+ * Copyright (C) 2003  Red Hat, Inc.
+ * Copyright (C) 2003  Jonathan Blandford <jrb@alum.mit.edu>
+ *
+ * Licensed under the Academic Free License version 2.0
+ * Or under the following terms:
+ * 
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 02111-1307, USA.
+ */
+
+
+#ifndef __XDG_MIME_H__
+#define __XDG_MIME_H__
+
+#include <stdlib.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif /* __cplusplus */
+
+#ifdef XDG_PREFIX
+#define XDG_ENTRY(func) _XDG_ENTRY2(XDG_PREFIX,func)
+#define _XDG_ENTRY2(prefix,func) _XDG_ENTRY3(prefix,func)
+#define _XDG_ENTRY3(prefix,func) prefix##_##func
+#endif
+
+typedef void (*XdgMimeCallback) (void *user_data);
+typedef void (*XdgMimeDestroy)  (void *user_data);
+
+  
+#ifdef XDG_PREFIX
+#define xdg_mime_get_mime_type_for_data       XDG_ENTRY(get_mime_type_for_data)
+#define xdg_mime_get_mime_type_for_file       XDG_ENTRY(get_mime_type_for_file)
+#define xdg_mime_get_mime_type_from_file_name XDG_ENTRY(get_mime_type_from_file_name)
+#define xdg_mime_is_valid_mime_type           XDG_ENTRY(is_valid_mime_type)
+#define xdg_mime_mime_type_equal              XDG_ENTRY(mime_type_equal)
+#define xdg_mime_media_type_equal             XDG_ENTRY(media_type_equal)
+#define xdg_mime_mime_type_subclass           XDG_ENTRY(mime_type_subclass)
+#define xdg_mime_get_mime_parents             XDG_ENTRY(get_mime_parents)
+#define xdg_mime_unalias_mime_type            XDG_ENTRY(unalias_mime_type)
+#define xdg_mime_get_max_buffer_extents       XDG_ENTRY(get_max_buffer_extents)
+#define xdg_mime_shutdown                     XDG_ENTRY(shutdown)
+#define xdg_mime_register_reload_callback     XDG_ENTRY(register_reload_callback)
+#define xdg_mime_remove_callback              XDG_ENTRY(remove_callback)
+#define xdg_mime_type_unknown                 XDG_ENTRY(type_unknown)
+#endif
+
+extern const char *xdg_mime_type_unknown;
+#define XDG_MIME_TYPE_UNKNOWN xdg_mime_type_unknown
+
+const char  *xdg_mime_get_mime_type_for_data       (const void *data,
+						    size_t      len);
+const char  *xdg_mime_get_mime_type_for_file       (const char *file_name);
+const char  *xdg_mime_get_mime_type_from_file_name (const char *file_name);
+int          xdg_mime_is_valid_mime_type           (const char *mime_type);
+int          xdg_mime_mime_type_equal              (const char *mime_a,
+						    const char *mime_b);
+int          xdg_mime_media_type_equal             (const char *mime_a,
+						    const char *mime_b);
+int          xdg_mime_mime_type_subclass           (const char *mime_a,
+						    const char *mime_b);
+const char **xdg_mime_get_mime_parents		   (const char *mime);
+const char  *xdg_mime_unalias_mime_type		   (const char *mime);
+int          xdg_mime_get_max_buffer_extents       (void);
+void         xdg_mime_shutdown                     (void);
+void         xdg_mime_dump                         (void);
+int          xdg_mime_register_reload_callback     (XdgMimeCallback  callback,
+						    void            *data,
+						    XdgMimeDestroy   destroy);
+void         xdg_mime_remove_callback              (int              callback_id);
+
+#ifdef __cplusplus
+}
+#endif /* __cplusplus */
+
+#endif /* __XDG_MIME_H__ */

xdgmimealias.c

@@ -0,0 +1,184 @@
+/* -*- mode: C; c-file-style: "gnu" -*- */
+/* xdgmimealias.c: Private file.  Datastructure for storing the aliases.
+ *
+ * More info can be found at http://www.freedesktop.org/standards/
+ *
+ * Copyright (C) 2004  Red Hat, Inc.
+ * Copyright (C) 2004  Matthias Clasen <mclasen@redhat.com>
+ *
+ * Licensed under the Academic Free License version 2.0
+ * Or under the following terms:
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 02111-1307, USA.
+ */
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif
+
+#include "xdgmimealias.h"
+#include "xdgmimeint.h"
+#include <stdlib.h>
+#include <stdio.h>
+#include <assert.h>
+#include <string.h>
+#include <fnmatch.h>
+
+#ifndef	FALSE
+#define	FALSE	(0)
+#endif
+
+#ifndef	TRUE
+#define	TRUE	(!FALSE)
+#endif
+
+typedef struct XdgAlias XdgAlias;
+
+struct XdgAlias 
+{
+  char *alias;
+  char *mime_type;
+};
+
+struct XdgAliasList
+{
+  struct XdgAlias *aliases;
+  int n_aliases;
+};
+
+XdgAliasList *
+_xdg_mime_alias_list_new (void)
+{
+  XdgAliasList *list;
+
+  list = malloc (sizeof (XdgAliasList));
+
+  list->aliases = NULL;
+  list->n_aliases = 0;
+
+  return list;
+}
+
+void         
+_xdg_mime_alias_list_free (XdgAliasList *list)
+{
+  int i;
+
+  if (list->aliases)
+    {
+      for (i = 0; i < list->n_aliases; i++)
+	{
+	  free (list->aliases[i].alias);
+	  free (list->aliases[i].mime_type);
+	}
+      free (list->aliases);
+    }
+  free (list);
+}
+
+static int
+alias_entry_cmp (const void *v1, const void *v2)
+{
+  return strcmp (((XdgAlias *)v1)->alias, ((XdgAlias *)v2)->alias);
+}
+
+const char  *
+_xdg_mime_alias_list_lookup (XdgAliasList *list,
+			     const char   *alias)
+{
+  XdgAlias *entry;
+  XdgAlias key;
+
+  if (list->n_aliases > 0)
+    {
+      key.alias = (char *)alias;
+      key.mime_type = 0;
+
+      entry = bsearch (&key, list->aliases, list->n_aliases,
+		       sizeof (XdgAlias), alias_entry_cmp);
+      if (entry)
+        return entry->mime_type;
+    }
+
+  return NULL;
+}
+
+void
+_xdg_mime_alias_read_from_file (XdgAliasList *list,
+				const char   *file_name)
+{
+  FILE *file;
+  char line[255];
+  int alloc;
+
+  file = fopen (file_name, "r");
+
+  if (file == NULL)
+    return;
+
+  /* FIXME: Not UTF-8 safe.  Doesn't work if lines are greater than 255 chars.
+   * Blah */
+  alloc = list->n_aliases + 16;
+  list->aliases = realloc (list->aliases, alloc * sizeof (XdgAlias));
+  while (fgets (line, 255, file) != NULL)
+    {
+      char *sep;
+      if (line[0] == '#')
+	continue;
+
+      sep = strchr (line, ' ');
+      if (sep == NULL)
+	continue;
+      *(sep++) = '\000';
+      sep[strlen (sep) -1] = '\000';
+      if (list->n_aliases == alloc)
+	{
+	  alloc <<= 1;
+	  list->aliases = realloc (list->aliases, 
+				   alloc * sizeof (XdgAlias));
+	}
+      list->aliases[list->n_aliases].alias = strdup (line);
+      list->aliases[list->n_aliases].mime_type = strdup (sep);
+      list->n_aliases++;
+    }
+  list->aliases = realloc (list->aliases, 
+			   list->n_aliases * sizeof (XdgAlias));
+
+  fclose (file);  
+  
+  if (list->n_aliases > 1)
+    qsort (list->aliases, list->n_aliases, 
+           sizeof (XdgAlias), alias_entry_cmp);
+}
+
+
+void
+_xdg_mime_alias_list_dump (XdgAliasList *list)
+{
+  int i;
+
+  if (list->aliases)
+    {
+      for (i = 0; i < list->n_aliases; i++)
+	{
+	  printf ("%s %s\n", 
+		  list->aliases[i].alias,
+		  list->aliases[i].mime_type);
+	}
+    }
+}
+
+

xdgmimealias.h

@@ -0,0 +1,50 @@
+/* -*- mode: C; c-file-style: "gnu" -*- */
+/* xdgmimealias.h: Private file.  Datastructure for storing the aliases.
+ *
+ * More info can be found at http://www.freedesktop.org/standards/
+ *
+ * Copyright (C) 2004  Red Hat, Inc.
+ * Copyright (C) 200  Matthias Clasen <mclasen@redhat.com>
+ *
+ * Licensed under the Academic Free License version 2.0
+ * Or under the following terms:
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 02111-1307, USA.
+ */
+
+#ifndef __XDG_MIME_ALIAS_H__
+#define __XDG_MIME_ALIAS_H__
+
+#include "xdgmime.h"
+
+typedef struct XdgAliasList XdgAliasList;
+
+#ifdef XDG_PREFIX
+#define _xdg_mime_alias_read_from_file        XDG_ENTRY(alias_read_from_file)
+#define _xdg_mime_alias_list_new              XDG_ENTRY(alias_list_new)
+#define _xdg_mime_alias_list_free             XDG_ENTRY(alias_list_free)
+#define _xdg_mime_alias_list_lookup           XDG_ENTRY(alias_list_lookup)
+#endif
+
+void          _xdg_mime_alias_read_from_file (XdgAliasList *list,
+					      const char   *file_name);
+XdgAliasList *_xdg_mime_alias_list_new       (void);
+void          _xdg_mime_alias_list_free      (XdgAliasList *list);
+const char   *_xdg_mime_alias_list_lookup    (XdgAliasList *list,
+					      const char  *alias);
+void          _xdg_mime_alias_list_dump      (XdgAliasList *list);
+
+#endif /* __XDG_MIME_ALIAS_H__ */

xdgmimeglob.c

@@ -0,0 +1,472 @@
+/* -*- mode: C; c-file-style: "gnu" -*- */
+/* xdgmimeglob.c: Private file.  Datastructure for storing the globs.
+ *
+ * More info can be found at http://www.freedesktop.org/standards/
+ *
+ * Copyright (C) 2003  Red Hat, Inc.
+ * Copyright (C) 2003  Jonathan Blandford <jrb@alum.mit.edu>
+ *
+ * Licensed under the Academic Free License version 2.0
+ * Or under the following terms:
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 02111-1307, USA.
+ */
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif
+
+#include "xdgmimeglob.h"
+#include "xdgmimeint.h"
+#include <stdlib.h>
+#include <stdio.h>
+#include <assert.h>
+#include <string.h>
+#include <fnmatch.h>
+
+#ifndef	FALSE
+#define	FALSE	(0)
+#endif
+
+#ifndef	TRUE
+#define	TRUE	(!FALSE)
+#endif
+
+typedef struct XdgGlobHashNode XdgGlobHashNode;
+typedef struct XdgGlobList XdgGlobList;
+
+struct XdgGlobHashNode
+{
+  xdg_unichar_t character;
+  const char *mime_type;
+  XdgGlobHashNode *next;
+  XdgGlobHashNode *child;
+};
+struct XdgGlobList
+{
+  const char *data;
+  const char *mime_type;
+  XdgGlobList *next;
+};
+
+struct XdgGlobHash
+{
+  XdgGlobList *literal_list;
+  XdgGlobHashNode *simple_node;
+  XdgGlobList *full_list;
+};
+
+
+/* XdgGlobList
+ */
+static XdgGlobList *
+_xdg_glob_list_new (void)
+{
+  XdgGlobList *new_element;
+
+  new_element = calloc (1, sizeof (XdgGlobList));
+
+  return new_element;
+}
+
+/* Frees glob_list and all of it's children */
+static void
+_xdg_glob_list_free (XdgGlobList *glob_list)
+{
+  XdgGlobList *ptr, *next;
+
+  ptr = glob_list;
+
+  while (ptr != NULL)
+    {
+      next = ptr->next;
+
+      if (ptr->data)
+	free ((void *) ptr->data);
+      if (ptr->mime_type)
+	free ((void *) ptr->mime_type);
+      free (ptr);
+
+      ptr = next;
+    }
+}
+
+static XdgGlobList *
+_xdg_glob_list_append (XdgGlobList *glob_list,
+		       void        *data,
+		       const char  *mime_type)
+{
+  XdgGlobList *new_element;
+  XdgGlobList *tmp_element;
+
+  new_element = _xdg_glob_list_new ();
+  new_element->data = data;
+  new_element->mime_type = mime_type;
+  if (glob_list == NULL)
+    return new_element;
+
+  tmp_element = glob_list;
+  while (tmp_element->next != NULL)
+    tmp_element = tmp_element->next;
+
+  tmp_element->next = new_element;
+
+  return glob_list;
+}
+
+#if 0
+static XdgGlobList *
+_xdg_glob_list_prepend (XdgGlobList *glob_list,
+			void        *data,
+			const char  *mime_type)
+{
+  XdgGlobList *new_element;
+
+  new_element = _xdg_glob_list_new ();
+  new_element->data = data;
+  new_element->next = glob_list;
+  new_element->mime_type = mime_type;
+
+  return new_element;
+}
+#endif
+
+/* XdgGlobHashNode
+ */
+
+static XdgGlobHashNode *
+_xdg_glob_hash_node_new (void)
+{
+  XdgGlobHashNode *glob_hash_node;
+
+  glob_hash_node = calloc (1, sizeof (XdgGlobHashNode));
+
+  return glob_hash_node;
+}
+
+static void
+_xdg_glob_hash_node_dump (XdgGlobHashNode *glob_hash_node,
+			  int depth)
+{
+  int i;
+  for (i = 0; i < depth; i++)
+    printf (" ");
+
+  printf ("%c", (char)glob_hash_node->character);
+  if (glob_hash_node->mime_type)
+    printf (" - %s\n", glob_hash_node->mime_type);
+  else
+    printf ("\n");
+  if (glob_hash_node->child)
+    _xdg_glob_hash_node_dump (glob_hash_node->child, depth + 1);
+  if (glob_hash_node->next)
+    _xdg_glob_hash_node_dump (glob_hash_node->next, depth);
+}
+
+static XdgGlobHashNode *
+_xdg_glob_hash_insert_text (XdgGlobHashNode *glob_hash_node,
+			    const char      *text,
+			    const char      *mime_type)
+{
+  XdgGlobHashNode *node;
+  xdg_unichar_t character;
+
+  character = _xdg_utf8_to_ucs4 (text);
+
+  if ((glob_hash_node == NULL) ||
+      (character < glob_hash_node->character))
+    {
+      node = _xdg_glob_hash_node_new ();
+      node->character = character;
+      node->next = glob_hash_node;
+      glob_hash_node = node;
+    }
+  else if (character == glob_hash_node->character)
+    {
+      node = glob_hash_node;
+    }
+  else
+    {
+      XdgGlobHashNode *prev_node;
+      int found_node = FALSE;
+
+      /* Look for the first character of text in glob_hash_node, and insert it if we
+       * have to.*/
+      prev_node = glob_hash_node;
+      node = prev_node->next;
+
+      while (node != NULL)
+	{
+	  if (character < node->character)
+	    {
+	      node = _xdg_glob_hash_node_new ();
+	      node->character = character;
+	      node->next = prev_node->next;
+	      prev_node->next = node;
+
+	      found_node = TRUE;
+	      break;
+	    }
+	  else if (character == node->character)
+	    {
+	      found_node = TRUE;
+	      break;
+	    }
+	  prev_node = node;
+	  node = node->next;
+	}
+
+      if (! found_node)
+	{
+	  node = _xdg_glob_hash_node_new ();
+	  node->character = character;
+	  node->next = prev_node->next;
+	  prev_node->next = node;
+	}
+    }
+
+  text = _xdg_utf8_next_char (text);
+  if (*text == '\000')
+    {
+      node->mime_type = mime_type;
+    }
+  else
+    {
+      node->child = _xdg_glob_hash_insert_text (node->child, text, mime_type);
+    }
+  return glob_hash_node;
+}
+
+static const char *
+_xdg_glob_hash_node_lookup_file_name (XdgGlobHashNode *glob_hash_node,
+				      const char      *file_name,
+				      int              ignore_case)
+{
+  XdgGlobHashNode *node;
+  xdg_unichar_t character;
+
+  if (glob_hash_node == NULL)
+    return NULL;
+
+  character = _xdg_utf8_to_ucs4 (file_name);
+  if (ignore_case)
+    character = _xdg_ucs4_to_lower(character);
+
+  for (node = glob_hash_node; node && character >= node->character; node = node->next)
+    {
+      if (character == node->character)
+	{
+	  file_name = _xdg_utf8_next_char (file_name);
+	  if (*file_name == '\000')
+	    return node->mime_type;
+	  else
+	    return _xdg_glob_hash_node_lookup_file_name (node->child,
+							 file_name,
+							 ignore_case);
+	}
+    }
+  return NULL;
+}
+
+const char *
+_xdg_glob_hash_lookup_file_name (XdgGlobHash *glob_hash,
+				 const char  *file_name)
+{
+  XdgGlobList *list;
+  const char *mime_type;
+  const char *ptr;
+  /* First, check the literals */
+
+  assert (file_name != NULL);
+
+  for (list = glob_hash->literal_list; list; list = list->next)
+    if (strcmp ((const char *)list->data, file_name) == 0)
+      return list->mime_type;
+
+  ptr = strchr (file_name, '.');
+  while (ptr != NULL)
+    {
+      mime_type = (_xdg_glob_hash_node_lookup_file_name (glob_hash->simple_node, ptr, FALSE));
+      if (mime_type != NULL)
+        return mime_type;
+      
+      mime_type = (_xdg_glob_hash_node_lookup_file_name (glob_hash->simple_node, ptr, TRUE));
+      if (mime_type != NULL)
+        return mime_type;
+      
+      ptr = strchr (ptr+1, '.');
+    }
+
+  /* FIXME: Not UTF-8 safe */
+  for (list = glob_hash->full_list; list; list = list->next)
+    if (fnmatch ((const char *)list->data, file_name, 0) == 0)
+      return list->mime_type;
+
+  return NULL;
+}
+
+
+
+/* XdgGlobHash
+ */
+
+XdgGlobHash *
+_xdg_glob_hash_new (void)
+{
+  XdgGlobHash *glob_hash;
+
+  glob_hash = calloc (1, sizeof (XdgGlobHash));
+
+  return glob_hash;
+}
+
+
+static void
+_xdg_glob_hash_free_nodes (XdgGlobHashNode *node)
+{
+  if (node)
+    {
+      if (node->child)
+       _xdg_glob_hash_free_nodes (node->child);
+      if (node->next)
+       _xdg_glob_hash_free_nodes (node->next);
+      if (node->mime_type)
+	free ((void *) node->mime_type);
+      free (node);
+    }
+}
+
+void
+_xdg_glob_hash_free (XdgGlobHash *glob_hash)
+{
+  _xdg_glob_list_free (glob_hash->literal_list);
+  _xdg_glob_list_free (glob_hash->full_list);
+  _xdg_glob_hash_free_nodes (glob_hash->simple_node);
+  free (glob_hash);
+}
+
+XdgGlobType
+_xdg_glob_determine_type (const char *glob)
+{
+  const char *ptr;
+  int maybe_in_simple_glob = FALSE;
+  int first_char = TRUE;
+
+  ptr = glob;
+
+  while (*ptr != '\000')
+    {
+      if (*ptr == '*' && first_char)
+	maybe_in_simple_glob = TRUE;
+      else if (*ptr == '\\' || *ptr == '[' || *ptr == '?' || *ptr == '*')
+	  return XDG_GLOB_FULL;
+
+      first_char = FALSE;
+      ptr = _xdg_utf8_next_char (ptr);
+    }
+  if (maybe_in_simple_glob)
+    return XDG_GLOB_SIMPLE;
+  else
+    return XDG_GLOB_LITERAL;
+}
+
+/* glob must be valid UTF-8 */
+void
+_xdg_glob_hash_append_glob (XdgGlobHash *glob_hash,
+			    const char  *glob,
+			    const char  *mime_type)
+{
+  XdgGlobType type;
+
+  assert (glob_hash != NULL);
+  assert (glob != NULL);
+
+  type = _xdg_glob_determine_type (glob);
+
+  switch (type)
+    {
+    case XDG_GLOB_LITERAL:
+      glob_hash->literal_list = _xdg_glob_list_append (glob_hash->literal_list, strdup (glob), strdup (mime_type));
+      break;
+    case XDG_GLOB_SIMPLE:
+      glob_hash->simple_node = _xdg_glob_hash_insert_text (glob_hash->simple_node, glob + 1, strdup (mime_type));
+      break;
+    case XDG_GLOB_FULL:
+      glob_hash->full_list = _xdg_glob_list_append (glob_hash->full_list, strdup (glob), strdup (mime_type));
+      break;
+    }
+}
+
+void
+_xdg_glob_hash_dump (XdgGlobHash *glob_hash)
+{
+  XdgGlobList *list;
+  printf ("LITERAL STRINGS\n");
+  if (glob_hash->literal_list == NULL)
+    {
+      printf ("    None\n");
+    }
+  else
+    {
+      for (list = glob_hash->literal_list; list; list = list->next)
+	printf ("    %s - %s\n", (char *)list->data, list->mime_type);
+    }
+  printf ("\nSIMPLE GLOBS\n");
+  _xdg_glob_hash_node_dump (glob_hash->simple_node, 4);
+
+  printf ("\nFULL GLOBS\n");
+  if (glob_hash->full_list == NULL)
+    {
+      printf ("    None\n");
+    }
+  else
+    {
+      for (list = glob_hash->full_list; list; list = list->next)
+	printf ("    %s - %s\n", (char *)list->data, list->mime_type);
+    }
+}
+
+
+void
+_xdg_mime_glob_read_from_file (XdgGlobHash *glob_hash,
+			       const char  *file_name)
+{
+  FILE *glob_file;
+  char line[255];
+
+  glob_file = fopen (file_name, "r");
+
+  if (glob_file == NULL)
+    return;
+
+  /* FIXME: Not UTF-8 safe.  Doesn't work if lines are greater than 255 chars.
+   * Blah */
+  while (fgets (line, 255, glob_file) != NULL)
+    {
+      char *colon;
+      if (line[0] == '#')
+	continue;
+
+      colon = strchr (line, ':');
+      if (colon == NULL)
+	continue;
+      *(colon++) = '\000';
+      colon[strlen (colon) -1] = '\000';
+      _xdg_glob_hash_append_glob (glob_hash, colon, line);
+    }
+
+  fclose (glob_file);
+}

xdgmimeglob.h

@@ -0,0 +1,65 @@
+/* -*- mode: C; c-file-style: "gnu" -*- */
+/* xdgmimeglob.h: Private file.  Datastructure for storing the globs.
+ *
+ * More info can be found at http://www.freedesktop.org/standards/
+ *
+ * Copyright (C) 2003  Red Hat, Inc.
+ * Copyright (C) 2003  Jonathan Blandford <jrb@alum.mit.edu>
+ *
+ * Licensed under the Academic Free License version 2.0
+ * Or under the following terms:
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 02111-1307, USA.
+ */
+
+#ifndef __XDG_MIME_GLOB_H__
+#define __XDG_MIME_GLOB_H__
+
+#include "xdgmime.h"
+
+typedef struct XdgGlobHash XdgGlobHash;
+
+typedef enum
+{
+  XDG_GLOB_LITERAL, /* Makefile */
+  XDG_GLOB_SIMPLE,  /* *.gif */
+  XDG_GLOB_FULL     /* x*.[ch] */
+} XdgGlobType;
+
+  
+#ifdef XDG_PREFIX
+#define _xdg_mime_glob_read_from_file         XDG_ENTRY(glob_read_from_file)
+#define _xdg_glob_hash_new                    XDG_ENTRY(hash_new)
+#define _xdg_glob_hash_free                   XDG_ENTRY(hash_free)
+#define _xdg_glob_hash_lookup_file_name       XDG_ENTRY(hash_lookup_file_name)
+#define _xdg_glob_hash_append_glob            XDG_ENTRY(hash_append_glob)
+#define _xdg_glob_determine_type              XDG_ENTRY(determine_type)
+#define _xdg_glob_hash_dump                   XDG_ENTRY(hash_dump)
+#endif
+
+void         _xdg_mime_glob_read_from_file   (XdgGlobHash *glob_hash,
+					      const char  *file_name);
+XdgGlobHash *_xdg_glob_hash_new              (void);
+void         _xdg_glob_hash_free             (XdgGlobHash *glob_hash);
+const char  *_xdg_glob_hash_lookup_file_name (XdgGlobHash *glob_hash,
+					      const char  *text);
+void         _xdg_glob_hash_append_glob      (XdgGlobHash *glob_hash,
+					      const char  *glob,
+					      const char  *mime_type);
+XdgGlobType  _xdg_glob_determine_type        (const char  *glob);
+void         _xdg_glob_hash_dump             (XdgGlobHash *glob_hash);
+
+#endif /* __XDG_MIME_GLOB_H__ */

xdgmimeint.c

@@ -0,0 +1,154 @@
+/* -*- mode: C; c-file-style: "gnu" -*- */
+/* xdgmimeint.c: Internal defines and functions.
+ *
+ * More info can be found at http://www.freedesktop.org/standards/
+ *
+ * Copyright (C) 2003  Red Hat, Inc.
+ * Copyright (C) 2003  Jonathan Blandford <jrb@alum.mit.edu>
+ *
+ * Licensed under the Academic Free License version 2.0
+ * Or under the following terms:
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 02111-1307, USA.
+ */
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif
+
+#include "xdgmimeint.h"
+#include <ctype.h>
+#include <string.h>
+
+#ifndef	FALSE
+#define	FALSE	(0)
+#endif
+
+#ifndef	TRUE
+#define	TRUE	(!FALSE)
+#endif
+
+static const unsigned char _xdg_utf8_skip_data[256] = {
+  1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+  1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+  1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+  1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+  1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+  1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
+  2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,
+  3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,5,5,5,5,6,6,1,1
+};
+
+const char * const _xdg_utf8_skip = _xdg_utf8_skip_data;
+
+
+
+/* Returns the number of unprocessed characters. */
+xdg_unichar_t
+_xdg_utf8_to_ucs4(const char *source)
+{
+  xdg_unichar_t ucs32;
+  if( ! ( *source & 0x80 ) )
+    {
+      ucs32 = *source;
+    }
+  else
+    {
+      int bytelength = 0;
+      xdg_unichar_t result;
+      if ( ! (*source & 0x40) )
+	{
+	  ucs32 = *source;
+	}
+      else
+	{
+	  if ( ! (*source & 0x20) )
+	    {
+	      result = *source++ & 0x1F;
+	      bytelength = 2;
+	    }
+	  else if ( ! (*source & 0x10) )
+	    {
+	      result = *source++ & 0x0F;
+	      bytelength = 3;
+	    }
+	  else if ( ! (*source & 0x08) )
+	    {
+	      result = *source++ & 0x07;
+	      bytelength = 4;
+	    }
+	  else if ( ! (*source & 0x04) )
+	    {
+	      result = *source++ & 0x03;
+	      bytelength = 5;
+	    }
+	  else if ( ! (*source & 0x02) )
+	    {
+	      result = *source++ & 0x01;
+	      bytelength = 6;
+	    }
+	  else
+	    {
+	      result = *source++;
+	      bytelength = 1;
+	    }
+
+	  for ( bytelength --; bytelength > 0; bytelength -- )
+	    {
+	      result <<= 6;
+	      result |= *source++ & 0x3F;
+	    }
+	  ucs32 = result;
+	}
+    }
+  return ucs32;
+}
+
+
+/* hullo.  this is great code.  don't rewrite it */
+
+xdg_unichar_t
+_xdg_ucs4_to_lower (xdg_unichar_t source)
+{
+  /* FIXME: Do a real to_upper sometime */
+  /* CaseFolding-3.2.0.txt has a table of rules. */
+  if ((source & 0xFF) == source)
+    return (xdg_unichar_t) tolower ((unsigned char) source);
+  return source;
+}
+
+int
+_xdg_utf8_validate (const char *source)
+{
+  /* FIXME: actually write */
+  return TRUE;
+}
+
+const char *
+_xdg_get_base_name (const char *file_name)
+{
+  const char *base_name;
+
+  if (file_name == NULL)
+    return NULL;
+
+  base_name = strrchr (file_name, '/');
+
+  if (base_name == NULL)
+    return file_name;
+  else
+    return base_name + 1;
+}

xdgmimeint.h

@@ -0,0 +1,73 @@
+/* -*- mode: C; c-file-style: "gnu" -*- */
+/* xdgmimeint.h: Internal defines and functions.
+ *
+ * More info can be found at http://www.freedesktop.org/standards/
+ *
+ * Copyright (C) 2003  Red Hat, Inc.
+ * Copyright (C) 2003  Jonathan Blandford <jrb@alum.mit.edu>
+ *
+ * Licensed under the Academic Free License version 2.0
+ * Or under the following terms:
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 02111-1307, USA.
+ */
+
+#ifndef __XDG_MIME_INT_H__
+#define __XDG_MIME_INT_H__
+
+#include "xdgmime.h"
+
+
+#ifndef	FALSE
+#define	FALSE (0)
+#endif
+
+#ifndef	TRUE
+#define	TRUE (!FALSE)
+#endif
+
+/* FIXME: Needs to be configure check */
+typedef unsigned int   xdg_unichar_t;
+typedef unsigned char  xdg_uchar8_t;
+typedef unsigned short xdg_uint16_t;
+typedef unsigned int   xdg_uint32_t;
+
+#ifdef XDG_PREFIX
+#define _xdg_utf8_skip   XDG_ENTRY(utf8_skip)
+#define _xdg_utf8_to_ucs4   XDG_ENTRY(utf8_to_ucs4)
+#define _xdg_ucs4_to_lower   XDG_ENTRY(ucs4_to_lower)
+#define _xdg_utf8_validate   XDG_ENTRY(utf8_validate)
+#define _xdg_get_base_name   XDG_ENTRY(get_ase_name)
+#endif
+
+#define SWAP_BE16_TO_LE16(val) (xdg_uint16_t)(((xdg_uint16_t)(val) << 8)|((xdg_uint16_t)(val) >> 8))
+
+#define SWAP_BE32_TO_LE32(val) (xdg_uint32_t)((((xdg_uint32_t)(val) & 0xFF000000U) >> 24) |	\
+					      (((xdg_uint32_t)(val) & 0x00FF0000U) >> 8) |	\
+					      (((xdg_uint32_t)(val) & 0x0000FF00U) << 8) |	\
+					      (((xdg_uint32_t)(val) & 0x000000FFU) << 24))
+/* UTF-8 utils
+ */
+extern const char *const _xdg_utf8_skip;
+#define _xdg_utf8_next_char(p) (char *)((p) + _xdg_utf8_skip[*(unsigned char *)(p)])
+#define _xdg_utf8_char_size(p) (int) (_xdg_utf8_skip[*(unsigned char *)(p)])
+
+xdg_unichar_t  _xdg_utf8_to_ucs4  (const char    *source);
+xdg_unichar_t  _xdg_ucs4_to_lower (xdg_unichar_t  source);
+int            _xdg_utf8_validate (const char    *source);
+const char    *_xdg_get_base_name (const char    *file_name);
+
+#endif /* __XDG_MIME_INT_H__ */

xdgmimemagic.c

@@ -0,0 +1,781 @@
+/* -*- mode: C; c-file-style: "gnu" -*- */
+/* xdgmimemagic.: Private file.  Datastructure for storing magic files.
+ *
+ * More info can be found at http://www.freedesktop.org/standards/
+ *
+ * Copyright (C) 2003  Red Hat, Inc.
+ * Copyright (C) 2003  Jonathan Blandford <jrb@alum.mit.edu>
+ *
+ * Licensed under the Academic Free License version 2.0
+ * Or under the following terms:
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 02111-1307, USA.
+ */
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif
+
+#include <assert.h>
+#include "xdgmimemagic.h"
+#include "xdgmimeint.h"
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <ctype.h>
+#include <errno.h>
+#include <limits.h>
+
+#ifndef	FALSE
+#define	FALSE	(0)
+#endif
+
+#ifndef	TRUE
+#define	TRUE	(!FALSE)
+#endif
+
+extern int errno;
+
+typedef struct XdgMimeMagicMatch XdgMimeMagicMatch;
+typedef struct XdgMimeMagicMatchlet XdgMimeMagicMatchlet;
+
+typedef enum
+{
+  XDG_MIME_MAGIC_SECTION,
+  XDG_MIME_MAGIC_MAGIC,
+  XDG_MIME_MAGIC_ERROR,
+  XDG_MIME_MAGIC_EOF
+} XdgMimeMagicState;
+
+struct XdgMimeMagicMatch
+{
+  const char *mime_type;
+  int priority;
+  XdgMimeMagicMatchlet *matchlet;
+  XdgMimeMagicMatch *next;
+};
+
+
+struct XdgMimeMagicMatchlet
+{
+  int indent;
+  int offset;
+  unsigned int value_length;
+  unsigned char *value;
+  unsigned char *mask;
+  unsigned int range_length;
+  unsigned int word_size;
+  XdgMimeMagicMatchlet *next;
+};
+
+
+struct XdgMimeMagic
+{
+  XdgMimeMagicMatch *match_list;
+  int max_extent;
+};
+
+static XdgMimeMagicMatch *
+_xdg_mime_magic_match_new (void)
+{
+  return calloc (1, sizeof (XdgMimeMagicMatch));
+}
+
+
+static XdgMimeMagicMatchlet *
+_xdg_mime_magic_matchlet_new (void)
+{
+  XdgMimeMagicMatchlet *matchlet;
+
+  matchlet = malloc (sizeof (XdgMimeMagicMatchlet));
+
+  matchlet->indent = 0;
+  matchlet->offset = 0;
+  matchlet->value_length = 0;
+  matchlet->value = NULL;
+  matchlet->mask = NULL;
+  matchlet->range_length = 1;
+  matchlet->word_size = 1;
+  matchlet->next = NULL;
+
+  return matchlet;
+}
+
+
+static void
+_xdg_mime_magic_matchlet_free (XdgMimeMagicMatchlet *mime_magic_matchlet)
+{
+  if (mime_magic_matchlet)
+    {
+      if (mime_magic_matchlet->next)
+	_xdg_mime_magic_matchlet_free (mime_magic_matchlet->next);
+      if (mime_magic_matchlet->value)
+	free (mime_magic_matchlet->value);
+      if (mime_magic_matchlet->mask)
+	free (mime_magic_matchlet->mask);
+      free (mime_magic_matchlet);
+    }
+}
+
+
+/* Frees mime_magic_match and the remainder of its list
+ */
+static void
+_xdg_mime_magic_match_free (XdgMimeMagicMatch *mime_magic_match)
+{
+  XdgMimeMagicMatch *ptr, *next;
+
+  ptr = mime_magic_match;
+  while (ptr)
+    {
+      next = ptr->next;
+
+      if (ptr->mime_type)
+	free ((void *) ptr->mime_type);
+      if (ptr->matchlet)
+	_xdg_mime_magic_matchlet_free (ptr->matchlet);
+      free (ptr);
+
+      ptr = next;
+    }
+}
+
+/* Reads in a hunk of data until a newline character or a '\000' is hit.  The
+ * returned string is null terminated, and doesn't include the newline.
+ */
+static unsigned char *
+_xdg_mime_magic_read_to_newline (FILE *magic_file,
+				 int  *end_of_file)
+{
+  unsigned char *retval;
+  int c;
+  int len, pos;
+
+  len = 128;
+  pos = 0;
+  retval = malloc (len);
+  *end_of_file = FALSE;
+
+  while (TRUE)
+    {
+      c = getc_unlocked (magic_file);
+      if (c == EOF)
+	{
+	  *end_of_file = TRUE;
+	  break;
+	}
+      if (c == '\n' || c == '\000')
+	break;
+      retval[pos++] = (unsigned char) c;
+      if (pos % 128 == 127)
+	{
+	  len = len + 128;
+	  retval = realloc (retval, len);
+	}
+    }
+
+  retval[pos] = '\000';
+  return retval;
+}
+
+/* Returns the number read from the file, or -1 if no number could be read.
+ */
+static int
+_xdg_mime_magic_read_a_number (FILE *magic_file,
+			       int  *end_of_file)
+{
+  /* LONG_MAX is about 20 characters on my system */
+#define MAX_NUMBER_SIZE 30
+  char number_string[MAX_NUMBER_SIZE + 1];
+  int pos = 0;
+  int c;
+  long retval = -1;
+
+  while (TRUE)
+    {
+      c = getc_unlocked (magic_file);
+
+      if (c == EOF)
+	{
+	  *end_of_file = TRUE;
+	  break;
+	}
+      if (! isdigit (c))
+	{
+	  ungetc (c, magic_file);
+	  break;
+	}
+      number_string[pos] = (char) c;
+      pos++;
+      if (pos == MAX_NUMBER_SIZE)
+	break;
+    }
+  if (pos > 0)
+    {
+      number_string[pos] = '\000';
+      errno = 0;
+      retval = strtol (number_string, NULL, 10);
+
+      if ((retval < INT_MIN) || (retval > INT_MAX) || (errno != 0))
+	return -1;
+    }
+
+  return retval;
+}
+
+/* Headers are of the format:
+ * [<priority>:<mime-type>]
+ */
+static XdgMimeMagicState
+_xdg_mime_magic_parse_header (FILE *magic_file, XdgMimeMagicMatch *match)
+{
+  int c;
+  char *buffer;
+  char *end_ptr;
+  int end_of_file = 0;
+
+  assert (magic_file != NULL);
+  assert (match != NULL);
+
+  c = getc_unlocked (magic_file);
+  if (c == EOF)
+    return XDG_MIME_MAGIC_EOF;
+  if (c != '[')
+    return XDG_MIME_MAGIC_ERROR;
+
+  match->priority = _xdg_mime_magic_read_a_number (magic_file, &end_of_file);
+  if (end_of_file)
+    return XDG_MIME_MAGIC_EOF;
+  if (match->priority == -1)
+    return XDG_MIME_MAGIC_ERROR;
+
+  c = getc_unlocked (magic_file);
+  if (c == EOF)
+    return XDG_MIME_MAGIC_EOF;
+  if (c != ':')
+    return XDG_MIME_MAGIC_ERROR;
+
+  buffer = _xdg_mime_magic_read_to_newline (magic_file, &end_of_file);
+  if (end_of_file)
+    return XDG_MIME_MAGIC_EOF;
+
+  end_ptr = buffer;
+  while (*end_ptr != ']' && *end_ptr != '\000' && *end_ptr != '\n')
+    end_ptr++;
+  if (*end_ptr != ']')
+    {
+      free (buffer);
+      return XDG_MIME_MAGIC_ERROR;
+    }
+  *end_ptr = '\000';
+
+  match->mime_type = strdup (buffer);
+  free (buffer);
+
+  return XDG_MIME_MAGIC_MAGIC;
+}
+
+static XdgMimeMagicState
+_xdg_mime_magic_parse_error (FILE *magic_file)
+{
+  int c;
+
+  while (1)
+    {
+      c = getc_unlocked (magic_file);
+      if (c == EOF)
+	return XDG_MIME_MAGIC_EOF;
+      if (c == '\n')
+	return XDG_MIME_MAGIC_SECTION;
+    }
+}
+
+/* Headers are of the format:
+ * [ indent ] ">" start-offset "=" value
+ * [ "&" mask ] [ "~" word-size ] [ "+" range-length ] "\n"
+ */
+static XdgMimeMagicState
+_xdg_mime_magic_parse_magic_line (FILE              *magic_file,
+				  XdgMimeMagicMatch *match)
+{
+  XdgMimeMagicMatchlet *matchlet;
+  int c;
+  int end_of_file;
+  int indent = 0;
+  int bytes_read;
+
+  assert (magic_file != NULL);
+
+  /* Sniff the buffer to make sure it's a valid line */
+  c = getc_unlocked (magic_file);
+  if (c == EOF)
+    return XDG_MIME_MAGIC_EOF;
+  else if (c == '[')
+    {
+      ungetc (c, magic_file);
+      return XDG_MIME_MAGIC_SECTION;
+    }
+  else if (c == '\n')
+    return XDG_MIME_MAGIC_MAGIC;
+
+  /* At this point, it must be a digit or a '>' */
+  end_of_file = FALSE;
+  if (isdigit (c))
+    {
+      ungetc (c, magic_file);
+      indent = _xdg_mime_magic_read_a_number (magic_file, &end_of_file);
+      if (end_of_file)
+	return XDG_MIME_MAGIC_EOF;
+      if (indent == -1)
+	return XDG_MIME_MAGIC_ERROR;
+      c = getc_unlocked (magic_file);
+      if (c == EOF)
+	return XDG_MIME_MAGIC_EOF;
+    }
+
+  if (c != '>')
+    return XDG_MIME_MAGIC_ERROR;
+
+  matchlet = _xdg_mime_magic_matchlet_new ();
+  matchlet->indent = indent;
+  matchlet->offset = _xdg_mime_magic_read_a_number (magic_file, &end_of_file);
+  if (end_of_file)
+    {
+      _xdg_mime_magic_matchlet_free (matchlet);
+      return XDG_MIME_MAGIC_EOF;
+    }
+  if (matchlet->offset == -1)
+    {
+      _xdg_mime_magic_matchlet_free (matchlet);
+      return XDG_MIME_MAGIC_ERROR;
+    }
+  c = getc_unlocked (magic_file);
+  if (c == EOF)
+    {
+      _xdg_mime_magic_matchlet_free (matchlet);
+      return XDG_MIME_MAGIC_EOF;
+    }
+  else if (c != '=')
+    {
+      _xdg_mime_magic_matchlet_free (matchlet);
+      return XDG_MIME_MAGIC_ERROR;
+    }
+
+  /* Next two bytes determine how long the value is */
+  matchlet->value_length = 0;
+  c = getc_unlocked (magic_file);
+  if (c == EOF)
+    {
+      _xdg_mime_magic_matchlet_free (matchlet);
+      return XDG_MIME_MAGIC_EOF;
+    }
+  matchlet->value_length = c & 0xFF;
+  matchlet->value_length = matchlet->value_length << 8;
+
+  c = getc_unlocked (magic_file);
+  if (c == EOF)
+    {
+      _xdg_mime_magic_matchlet_free (matchlet);
+      return XDG_MIME_MAGIC_EOF;
+    }
+  matchlet->value_length = matchlet->value_length + (c & 0xFF);
+
+  matchlet->value = malloc (matchlet->value_length);
+
+  /* OOM */
+  if (matchlet->value == NULL)
+    {
+      _xdg_mime_magic_matchlet_free (matchlet);
+      return XDG_MIME_MAGIC_ERROR;
+    }
+  bytes_read = fread (matchlet->value, 1, matchlet->value_length, magic_file);
+  if (bytes_read != matchlet->value_length)
+    {
+      _xdg_mime_magic_matchlet_free (matchlet);
+      if (feof (magic_file))
+	return XDG_MIME_MAGIC_EOF;
+      else
+	return XDG_MIME_MAGIC_ERROR;
+    }
+
+  c = getc_unlocked (magic_file);
+  if (c == '&')
+    {
+      matchlet->mask = malloc (matchlet->value_length);
+      /* OOM */
+      if (matchlet->mask == NULL)
+	{
+	  _xdg_mime_magic_matchlet_free (matchlet);
+	  return XDG_MIME_MAGIC_ERROR;
+	}
+      bytes_read = fread (matchlet->mask, 1, matchlet->value_length, magic_file);
+      if (bytes_read != matchlet->value_length)
+	{
+	  _xdg_mime_magic_matchlet_free (matchlet);
+	  if (feof (magic_file))
+	    return XDG_MIME_MAGIC_EOF;
+	  else
+	    return XDG_MIME_MAGIC_ERROR;
+	}
+      c = getc_unlocked (magic_file);
+    }
+
+  if (c == '~')
+    {
+      matchlet->word_size = _xdg_mime_magic_read_a_number (magic_file, &end_of_file);
+      if (end_of_file)
+	{
+	  _xdg_mime_magic_matchlet_free (matchlet);
+	  return XDG_MIME_MAGIC_EOF;
+	}
+      if (matchlet->word_size != 0 &&
+	  matchlet->word_size != 1 &&
+	  matchlet->word_size != 2 &&
+	  matchlet->word_size != 4)
+	{
+	  _xdg_mime_magic_matchlet_free (matchlet);
+	  return XDG_MIME_MAGIC_ERROR;
+	}
+      c = getc_unlocked (magic_file);
+    }
+
+  if (c == '+')
+    {
+      matchlet->range_length = _xdg_mime_magic_read_a_number (magic_file, &end_of_file);
+      if (end_of_file)
+	{
+	  _xdg_mime_magic_matchlet_free (matchlet);
+	  return XDG_MIME_MAGIC_EOF;
+	}
+      if (matchlet->range_length == -1)
+	{
+	  _xdg_mime_magic_matchlet_free (matchlet);
+	  return XDG_MIME_MAGIC_ERROR;
+	}
+      c = getc_unlocked (magic_file);
+    }
+
+
+  if (c == '\n')
+    {
+      /* We clean up the matchlet, byte swapping if needed */
+      if (matchlet->word_size > 1)
+	{
+	  int i;
+	  if (matchlet->value_length % matchlet->word_size != 0)
+	    {
+	      _xdg_mime_magic_matchlet_free (matchlet);
+	      return XDG_MIME_MAGIC_ERROR;
+	    }
+	  /* FIXME: need to get this defined in a <config.h> style file */
+#if LITTLE_ENDIAN
+	  for (i = 0; i < matchlet->value_length; i = i + matchlet->word_size)
+	    {
+	      if (matchlet->word_size == 2)
+		*((xdg_uint16_t *) matchlet->value + i) = SWAP_BE16_TO_LE16 (*((xdg_uint16_t *) (matchlet->value + i)));
+	      else if (matchlet->word_size == 4)
+		*((xdg_uint32_t *) matchlet->value + i) = SWAP_BE32_TO_LE32 (*((xdg_uint32_t *) (matchlet->value + i)));
+	      if (matchlet->mask)
+		{
+		  if (matchlet->word_size == 2)
+		    *((xdg_uint16_t *) matchlet->mask + i) = SWAP_BE16_TO_LE16 (*((xdg_uint16_t *) (matchlet->mask + i)));
+		  else if (matchlet->word_size == 4)
+		    *((xdg_uint32_t *) matchlet->mask + i) = SWAP_BE32_TO_LE32 (*((xdg_uint32_t *) (matchlet->mask + i)));
+
+		}
+	    }
+#endif
+	}
+
+      matchlet->next = match->matchlet;
+      match->matchlet = matchlet;
+
+
+      return XDG_MIME_MAGIC_MAGIC;
+    }
+
+  _xdg_mime_magic_matchlet_free (matchlet);
+  if (c == EOF)
+    return XDG_MIME_MAGIC_EOF;
+
+  return XDG_MIME_MAGIC_ERROR;
+}
+
+static int
+_xdg_mime_magic_matchlet_compare_to_data (XdgMimeMagicMatchlet *matchlet,
+					  const void           *data,
+					  size_t                len)
+{
+  int i, j;
+
+  for (i = matchlet->offset; i <= matchlet->offset + matchlet->range_length; i++)
+    {
+      int valid_matchlet = TRUE;
+
+      if (i + matchlet->value_length > len)
+	return FALSE;
+
+      if (matchlet->mask)
+	{
+	  for (j = 0; j < matchlet->value_length; j++)
+	    {
+	      if ((matchlet->value[j] & matchlet->mask[j]) !=
+		  ((((unsigned char *) data)[j + i]) & matchlet->mask[j]))
+		{
+		  valid_matchlet = FALSE;
+		  break;
+		}
+	    }
+	}
+      else
+	{
+	  for (j = 0; j <  matchlet->value_length; j++)
+	    {
+	      if (matchlet->value[j] != ((unsigned char *) data)[j + i])
+		{
+		  valid_matchlet = FALSE;
+		  break;
+		}
+	    }
+	}
+      if (valid_matchlet)
+	return TRUE;
+    }
+  return FALSE;
+}
+
+static int
+_xdg_mime_magic_matchlet_compare_level (XdgMimeMagicMatchlet *matchlet,
+					const void           *data,
+					size_t                len,
+					int                   indent)
+{
+  while ((matchlet != NULL) && (matchlet->indent == indent))
+    {
+      if (_xdg_mime_magic_matchlet_compare_to_data (matchlet, data, len))
+	{
+	  if ((matchlet->next == NULL) ||
+	      (matchlet->next->indent <= indent))
+	    return TRUE;
+
+	  if (_xdg_mime_magic_matchlet_compare_level (matchlet->next,
+						      data,
+						      len,
+						      indent + 1))
+	    return TRUE;
+	}
+
+      do
+	{
+	  matchlet = matchlet->next;
+	}
+      while (matchlet && matchlet->indent > indent);
+    }
+
+  return FALSE;
+}
+
+static int
+_xdg_mime_magic_match_compare_to_data (XdgMimeMagicMatch *match,
+				       const void        *data,
+				       size_t             len)
+{
+  return _xdg_mime_magic_matchlet_compare_level (match->matchlet, data, len, 0);
+}
+
+static void
+_xdg_mime_magic_insert_match (XdgMimeMagic      *mime_magic,
+			      XdgMimeMagicMatch *match)
+{
+  XdgMimeMagicMatch *list;
+
+  if (mime_magic->match_list == NULL)
+    {
+      mime_magic->match_list = match;
+      return;
+    }
+
+  if (match->priority > mime_magic->match_list->priority)
+    {
+      match->next = mime_magic->match_list;
+      mime_magic->match_list = match;
+      return;
+    }
+
+  list = mime_magic->match_list;
+  while (list->next != NULL)
+    {
+      if (list->next->priority < match->priority)
+	{
+	  match->next = list->next;
+	  list->next = match;
+	  return;
+	}
+      list = list->next;
+    }
+  list->next = match;
+  match->next = NULL;
+}
+
+XdgMimeMagic *
+_xdg_mime_magic_new (void)
+{
+  return calloc (1, sizeof (XdgMimeMagic));
+}
+
+void
+_xdg_mime_magic_free (XdgMimeMagic *mime_magic)
+{
+  if (mime_magic) {
+    _xdg_mime_magic_match_free (mime_magic->match_list);
+    free (mime_magic);
+  }
+}
+
+int
+_xdg_mime_magic_get_buffer_extents (XdgMimeMagic *mime_magic)
+{
+  return mime_magic->max_extent;
+}
+
+const char *
+_xdg_mime_magic_lookup_data (XdgMimeMagic *mime_magic,
+			     const void   *data,
+			     size_t        len)
+{
+  XdgMimeMagicMatch *match;
+
+  for (match = mime_magic->match_list; match; match = match->next)
+    {
+      if (_xdg_mime_magic_match_compare_to_data (match, data, len))
+	{
+	  return match->mime_type;
+	}
+    }
+
+  return NULL;
+}
+
+static void
+_xdg_mime_update_mime_magic_extents (XdgMimeMagic *mime_magic)
+{
+  XdgMimeMagicMatch *match;
+  int max_extent = 0;
+
+  for (match = mime_magic->match_list; match; match = match->next)
+    {
+      XdgMimeMagicMatchlet *matchlet;
+
+      for (matchlet = match->matchlet; matchlet; matchlet = matchlet->next)
+	{
+	  int extent;
+
+	  extent = matchlet->value_length + matchlet->offset + matchlet->range_length;
+	  if (max_extent < extent)
+	    max_extent = extent;
+	}
+    }
+
+  mime_magic->max_extent = max_extent;
+}
+
+static XdgMimeMagicMatchlet *
+_xdg_mime_magic_matchlet_mirror (XdgMimeMagicMatchlet *matchlets)
+{
+  XdgMimeMagicMatchlet *new_list;
+  XdgMimeMagicMatchlet *tmp;
+
+  if ((matchlets == NULL) || (matchlets->next == NULL))
+    return matchlets;
+
+  new_list = NULL;
+  tmp = matchlets;
+  while (tmp != NULL)
+    {
+      XdgMimeMagicMatchlet *matchlet;
+
+      matchlet = tmp;
+      tmp = tmp->next;
+      matchlet->next = new_list;
+      new_list = matchlet;
+    }
+
+  return new_list;
+
+}
+
+static void
+_xdg_mime_magic_read_magic_file (XdgMimeMagic *mime_magic,
+				 FILE         *magic_file)
+{
+  XdgMimeMagicState state;
+  XdgMimeMagicMatch *match = NULL; /* Quiet compiler */
+
+  state = XDG_MIME_MAGIC_SECTION;
+
+  while (state != XDG_MIME_MAGIC_EOF)
+    {
+      switch (state)
+	{
+	case XDG_MIME_MAGIC_SECTION:
+	  match = _xdg_mime_magic_match_new ();
+	  state = _xdg_mime_magic_parse_header (magic_file, match);
+	  if (state == XDG_MIME_MAGIC_EOF || state == XDG_MIME_MAGIC_ERROR)
+	    _xdg_mime_magic_match_free (match);
+	  break;
+	case XDG_MIME_MAGIC_MAGIC:
+	  state = _xdg_mime_magic_parse_magic_line (magic_file, match);
+	  if (state == XDG_MIME_MAGIC_SECTION ||
+	      (state == XDG_MIME_MAGIC_EOF && match->mime_type))
+	    {
+	      match->matchlet = _xdg_mime_magic_matchlet_mirror (match->matchlet);
+	      _xdg_mime_magic_insert_match (mime_magic, match);
+	    }
+	  else if (state == XDG_MIME_MAGIC_EOF || state == XDG_MIME_MAGIC_ERROR)
+	    _xdg_mime_magic_match_free (match);
+	  break;
+	case XDG_MIME_MAGIC_ERROR:
+	  state = _xdg_mime_magic_parse_error (magic_file);
+	  break;
+	case XDG_MIME_MAGIC_EOF:
+	default:
+	  /* Make the compiler happy */
+	  assert (0);
+	}
+    }
+  _xdg_mime_update_mime_magic_extents (mime_magic);
+}
+
+void
+_xdg_mime_magic_read_from_file (XdgMimeMagic *mime_magic,
+				const char   *file_name)
+{
+  FILE *magic_file;
+  char header[12];
+
+  magic_file = fopen (file_name, "r");
+
+  if (magic_file == NULL)
+    return;
+
+  if (fread (header, 1, 12, magic_file) == 12)
+    {
+      if (memcmp ("MIME-Magic\0\n", header, 12) == 0)
+        _xdg_mime_magic_read_magic_file (mime_magic, magic_file);
+    }
+
+  fclose (magic_file);
+}

xdgmimemagic.h

@@ -0,0 +1,54 @@
+/* -*- mode: C; c-file-style: "gnu" -*- */
+/* xdgmimemagic.h: Private file.  Datastructure for storing the magic files.
+ *
+ * More info can be found at http://www.freedesktop.org/standards/
+ *
+ * Copyright (C) 2003  Red Hat, Inc.
+ * Copyright (C) 2003  Jonathan Blandford <jrb@alum.mit.edu>
+ *
+ * Licensed under the Academic Free License version 2.0
+ * Or under the following terms:
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 02111-1307, USA.
+ */
+
+#ifndef __XDG_MIME_MAGIC_H__
+#define __XDG_MIME_MAGIC_H__
+
+#include <unistd.h>
+#include "xdgmime.h"
+typedef struct XdgMimeMagic XdgMimeMagic;
+
+#ifdef XDG_PREFIX
+#define _xdg_mime_glob_read_from_file             XDG_ENTRY(glob_read_from_file)
+#define _xdg_mime_magic_new                       XDG_ENTRY(magic_new)
+#define _xdg_mime_magic_read_from_file            XDG_ENTRY(magic_read_from_file)
+#define _xdg_mime_magic_free                      XDG_ENTRY(magic_free)
+#define _xdg_mime_magic_get_buffer_extents        XDG_ENTRY(magic_get_buffer_extents)
+#define _xdg_mime_magic_lookup_data               XDG_ENTRY(magic_lookup_data)
+#endif
+
+
+XdgMimeMagic *_xdg_mime_magic_new                (void);
+void          _xdg_mime_magic_read_from_file     (XdgMimeMagic *mime_magic,
+						  const char   *file_name);
+void          _xdg_mime_magic_free               (XdgMimeMagic *mime_magic);
+int           _xdg_mime_magic_get_buffer_extents (XdgMimeMagic *mime_magic);
+const char   *_xdg_mime_magic_lookup_data        (XdgMimeMagic *mime_magic,
+						  const void   *data,
+						  size_t        len);
+
+#endif /* __XDG_MIME_MAGIC_H__ */

xdgmimeparent.c

@@ -0,0 +1,219 @@
+/* -*- mode: C; c-file-style: "gnu" -*- */
+/* xdgmimealias.c: Private file.  Datastructure for storing the hierarchy.
+ *
+ * More info can be found at http://www.freedesktop.org/standards/
+ *
+ * Copyright (C) 2004  Red Hat, Inc.
+ * Copyright (C) 2004  Matthias Clasen <mclasen@redhat.com>
+ *
+ * Licensed under the Academic Free License version 2.0
+ * Or under the following terms:
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 02111-1307, USA.
+ */
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif
+
+#include "xdgmimeparent.h"
+#include "xdgmimeint.h"
+#include <stdlib.h>
+#include <stdio.h>
+#include <assert.h>
+#include <string.h>
+#include <fnmatch.h>
+
+#ifndef	FALSE
+#define	FALSE	(0)
+#endif
+
+#ifndef	TRUE
+#define	TRUE	(!FALSE)
+#endif
+
+typedef struct XdgMimeParents XdgMimeParents;
+
+struct XdgMimeParents
+{
+  char *mime;
+  char **parents;
+  int n_parents;
+};
+
+struct XdgParentList
+{
+  struct XdgMimeParents *parents;
+  int n_mimes;
+};
+
+XdgParentList *
+_xdg_mime_parent_list_new (void)
+{
+  XdgParentList *list;
+
+  list = malloc (sizeof (XdgParentList));
+
+  list->parents = NULL;
+  list->n_mimes = 0;
+
+  return list;
+}
+
+void         
+_xdg_mime_parent_list_free (XdgParentList *list)
+{
+  int i;
+  char **p;
+
+  if (list->parents)
+    {
+      for (i = 0; i < list->n_mimes; i++)
+	{
+	  for (p = list->parents[i].parents; *p; p++)
+	    free (*p);
+
+	  free (list->parents[i].parents);
+	  free (list->parents[i].mime);
+	}
+      free (list->parents);
+    }
+  free (list);
+}
+
+static int
+parent_entry_cmp (const void *v1, const void *v2)
+{
+  return strcmp (((XdgMimeParents *)v1)->mime, ((XdgMimeParents *)v2)->mime);
+}
+
+const char **
+_xdg_mime_parent_list_lookup (XdgParentList *list,
+			      const char    *mime)
+{
+  XdgMimeParents *entry;
+  XdgMimeParents key;
+
+  if (list->n_mimes > 0)
+    {
+      key.mime = (char *)mime;
+      key.parents = NULL;
+
+      entry = bsearch (&key, list->parents, list->n_mimes,
+		       sizeof (XdgMimeParents), &parent_entry_cmp);
+      if (entry)
+        return (const char **)entry->parents;
+    }
+
+  return NULL;
+}
+
+void
+_xdg_mime_parent_read_from_file (XdgParentList *list,
+				 const char    *file_name)
+{
+  FILE *file;
+  char line[255];
+  int i, alloc;
+  XdgMimeParents *entry;
+
+  file = fopen (file_name, "r");
+
+  if (file == NULL)
+    return;
+
+  /* FIXME: Not UTF-8 safe.  Doesn't work if lines are greater than 255 chars.
+   * Blah */
+  alloc = list->n_mimes + 16;
+  list->parents = realloc (list->parents, alloc * sizeof (XdgMimeParents));
+  while (fgets (line, 255, file) != NULL)
+    {
+      char *sep;
+      if (line[0] == '#')
+	continue;
+
+      sep = strchr (line, ' ');
+      if (sep == NULL)
+	continue;
+      *(sep++) = '\000';
+      sep[strlen (sep) -1] = '\000';
+      entry = NULL;
+      for (i = 0; i < list->n_mimes; i++)
+	{
+	  if (strcmp (list->parents[i].mime, line) == 0)
+	    {
+	      entry = &(list->parents[i]);
+	      break;
+	    }
+	}
+      
+      if (!entry)
+	{
+	  if (list->n_mimes == alloc)
+	    {
+	      alloc <<= 1;
+	      list->parents = realloc (list->parents, 
+				       alloc * sizeof (XdgMimeParents));
+	    }
+	  list->parents[list->n_mimes].mime = strdup (line);
+	  list->parents[list->n_mimes].parents = NULL;
+	  entry = &(list->parents[list->n_mimes]);
+	  list->n_mimes++;
+	}
+
+      if (!entry->parents)
+	{
+	  entry->n_parents = 1;
+	  entry->parents = malloc ((entry->n_parents + 1) * sizeof (char *));
+	}
+      else
+	{
+	  entry->n_parents += 1;
+	  entry->parents = realloc (entry->parents, 
+				    (entry->n_parents + 2) * sizeof (char *));
+	}
+      entry->parents[entry->n_parents - 1] = strdup (sep);
+      entry->parents[entry->n_parents] = NULL;
+    }
+
+  list->parents = realloc (list->parents, 
+			   list->n_mimes * sizeof (XdgMimeParents));
+
+  fclose (file);  
+  
+  if (list->n_mimes > 1)
+    qsort (list->parents, list->n_mimes, 
+           sizeof (XdgMimeParents), &parent_entry_cmp);
+}
+
+
+void         
+_xdg_mime_parent_list_dump (XdgParentList *list)
+{
+  int i;
+  char **p;
+
+  if (list->parents)
+    {
+      for (i = 0; i < list->n_mimes; i++)
+	{
+	  for (p = list->parents[i].parents; *p; p++)
+	    printf ("%s %s\n", list->parents[i].mime, *p);
+	}
+    }
+}
+
+

xdgmimeparent.h

@@ -0,0 +1,50 @@
+/* -*- mode: C; c-file-style: "gnu" -*- */
+/* xdgmimeparent.h: Private file.  Datastructure for storing the hierarchy.
+ *
+ * More info can be found at http://www.freedesktop.org/standards/
+ *
+ * Copyright (C) 2004  Red Hat, Inc.
+ * Copyright (C) 200  Matthias Clasen <mclasen@redhat.com>
+ *
+ * Licensed under the Academic Free License version 2.0
+ * Or under the following terms:
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 02111-1307, USA.
+ */
+
+#ifndef __XDG_MIME_PARENT_H__
+#define __XDG_MIME_PARENT_H__
+
+#include "xdgmime.h"
+
+typedef struct XdgParentList XdgParentList;
+
+#ifdef XDG_PREFIX
+#define _xdg_mime_parent_read_from_file        XDG_ENTRY(parent_read_from_file)
+#define _xdg_mime_parent_list_new              XDG_ENTRY(parent_list_new)
+#define _xdg_mime_parent_list_free             XDG_ENTRY(parent_list_free)
+#define _xdg_mime_parent_list_lookup           XDG_ENTRY(parent_list_lookup)
+#endif
+
+void          _xdg_mime_parent_read_from_file (XdgParentList *list,
+					       const char    *file_name);
+XdgParentList *_xdg_mime_parent_list_new       (void);
+void           _xdg_mime_parent_list_free      (XdgParentList *list);
+const char   **_xdg_mime_parent_list_lookup    (XdgParentList *list,
+						const char    *mime);
+void           _xdg_mime_parent_list_dump      (XdgParentList *list);
+
+#endif /* __XDG_MIME_PARENT_H__ */