diff options
author | Flavio Cruz <flaviocruz@gmail.com> | 2008-08-15 14:58:01 +0000 |
---|---|---|
committer | Flavio Cruz <flaviocruz@gmail.com> | 2008-08-15 14:58:01 +0000 |
commit | ec8f753dc88b18e09c08c32e1ad95742cb46519f (patch) | |
tree | 6d8ad38af73b5049f6a2a09c7248bba95d72eb20 | |
parent | c3995e7410f6d7bd7e526d2635cefb8f7d549dc8 (diff) |
Convert to texi format. Remove txt file. Add Makefile.
-rw-r--r-- | docs/Makefile | 5 | ||||
-rw-r--r-- | docs/tutorial.texi (renamed from docs/tutorial.txt) | 538 |
2 files changed, 408 insertions, 135 deletions
diff --git a/docs/Makefile b/docs/Makefile new file mode 100644 index 000000000..6202605cc --- /dev/null +++ b/docs/Makefile @@ -0,0 +1,5 @@ + +all: tutorial.html + +tutorial.html: tutorial.texi + texi2html tutorial.texi diff --git a/docs/tutorial.txt b/docs/tutorial.texi index f45311094..1e1d7244e 100644 --- a/docs/tutorial.txt +++ b/docs/tutorial.texi @@ -1,79 +1,151 @@ +\input tutorial-lisp.tex +@setfilename tutorial-lisp.info +@settitle Lisp Hurd bindings + +@copying +Copyright (C) 2008 +Free Software Foundation, Inc. +@quotation +Permission is granted to copy, distribute and/or modify this document +under the terms of the GNU Free Documentation License, Version 1.1 or +any later version published by the Free Software Foundation; with no +Invariant Sections, with the Front-Cover texts being ``A GNU +Manual'', and with the Back-Cover Texts as in (a) below. A copy of the +license is included in the section entitled ``GNU Free Documentation +License'' in the Emacs manual. + +(a) The FSF's Back-Cover Text is: ``You have freedom to copy and modify +this GNU Manual, like GNU software. Copies published by the Free +Software Foundation raise funds for GNU development.'' + +This document is part of a collection distributed under the GNU Free +Documentation License. If you want to distribute this document +separately from the collection, you can do so by adding a copy of the +license to the document, as described in section 6 of the license. +@end quotation +@end copying + +@dircategory Programming Lisp translators +@direntry +* Lisp translators (lisp translators). How to program Lisp translators. +@end direntry + +@titlepage +@title Programming Lisp Translators +@author Flavio Cruz +@page +@insertcopying +@end titlepage + +@contents +@node Top +@top Introduction -==================================== - Using the Lisp translator library -==================================== +This document will introduce you to a new Hurd translator library that can be used to easily implement new file system servers. -0. Contents. +The main advantages of this library compared to libnetfs or libtrivfs, are: - 1. Introduction - 2. Dependencies - 3. Installation - 4. A /dev/zero like translator. - 5. A more complex example. - 6. Conclusion +@itemize @bullet +@item +A much more fast development - What can take weeks to do in libnetfs, can be done in a few hours or less. +@item +Easy development - You can put more focus on high level details of your translators and less on the little details that arise when writing C based translators. +@item +Use of the Common Lisp language - Programming in Lisp is great fun. +@item +Prototyping - You can test and validate new translator ideas and then build a fast implementation in C. +@end itemize +Of course, there are also some drawbacks: --------------------- -1. Introduction --------------------- +@itemize @bullet +@item +Efficiency - Don't expect to run your Lisp based translators as fast as the C based ones. +@item +Disk based translators - Don't expect to write the next generation ext file system. +@end itemize + +Given these pros and cons, it's easy to deduce that this library is more suited to implement virtual file systems. Examples are: +@itemize @bullet +@item +Translators were data is located in a local file (like zipfs, tarfs, rarfs, ...) +@item +Single file translators (that do content filtering, output of a command, etc). +@item +Network based file systems (ftpfs, httpfs, ircfs, ...) +@item +Proxy file systems (like hostmux, usermux, etc..). +@end itemize + +@menu +* Dependencies:: What needs to be installed before using the library. +* Installation:: How to install the library. +* A /dev/zero like translator:: Introductory example. +* A more complex example:: A more compreensive example. +* Conclusion:: Conclusion text. +@end menu + +@node Dependencies, Installation, Top, Top +@chapter Dependencies -This document will introduce you to a new Hurd translator library that can be used to easily implement new file system servers. +Right now, only CLISP is supported. Porting to new Lisp implementations will happen has soon they run/compile sanely on the Hurd operating system (especially SBCL ;-)). -The main advantages of this library compared to libnetfs or libtrivfs, are: +You must also have these asdf packages installed on your system: +@itemize @bullet +@item +@uref{http://www.common-lisp.net/project/cffi/,CFFI} - (1) - A much more fast development - What can take weeks to do in libnetfs, can be done in a few hours or less. - (2) - Easy development - You can put more focus on high level details of your translators and less on the little details that arise when writing C based translators. - (3) - Use of the Common Lisp language - Programming in Lisp is great fun. - (4) - Prototyping - You can test and validate new translator ideas and then build a fast implementation in C. +It can be installed on Debian GNU/Hurd with apt-get install cl-cffi. -Of course, there are also some drawbacks: +@item +@uref{http://www.weitz.de/flexi-streams/,flexi-streams} - (1) - Efficiency - Don't expect to run your Lisp based translators as fast as the C based ones. - (2) - Disk based translators - Don't expect to write the next generation ext file system. +Can also be installed with apt-get install cl-flexi-streams. -Given these pros and cons, it's easy to deduce that this library is more suited to implement virtual file systems. Examples are: translators were data is located in a local file (like zipfs, tarfs, rarfs, ...), single file translators (that do content filtering, output of a command, etc), network based file systems (ftpfs, httpfs, ircfs, ...), proxy file systems (like hostmux, usermux, etc..). +@item +@uref{http://www.cliki.net/trivial-garbage?v=20,trivial-garbage} -------------------------- -2. Dependencies -------------------------- +Not apt-get installable, but please grab the tarball from the project web page and install it this way: -Right now, only CLISP is supported. Porting to new Lisp implementations will happen has soon they run/compile sanely on the Hurd operating system (especially SBCL ;-)). +@example +# cd /usr/share/common-lisp/source +# tar zxvf <path to trivial_garbage.tar.gz> +# ln -sf $PWD/trivial-garbage_0.16/trivial-garbage.asd ../systems/ +@end example -You must also have these asdf packages installed on your system: - - CFFI: (http://www.common-lisp.net/project/cffi/). It can be installed on Debian GNU/Hurd with apt-get install cl-cffi. - - flexi-streams (http://www.weitz.de/flexi-streams/). Can also be installed with apt-get install cl-flexi-streams. - - trivial-garbage (http://www.cliki.net/trivial-garbage?v=20). Not apt-get installable, but please grab the tarball from the project web page and install it this way: - # cd /usr/share/common-lisp/source - # tar zxvf <path to trivial_garbage.tar.gz> - # ln -sf $PWD/trivial-garbage_0.16/trivial-garbage.asd ../systems/ +@end itemize ------------------------- -3. Installation ------------------------- +@node Installation, A /dev/zero like translator, Top, Top +@chapter Installation -To install the translator library, grab the Hurd source tree from the branch flaviocruz-soc2008-lisp-branch, and start building it: +To install the translator library, grab the Hurd source tree from the branch @code{flaviocruz-soc2008-lisp-branch} and build it: - # mkdir hurd-build - # cd hurd-build - # ../hurd/configure --prefix=/usr --enable-lisp - # make - # make install +@example -With everything in place I recommend you to previously compile everything. In the CLISP read-eval-print loop do: +# cvs -z3 -d:pserver:anonymous@@cvs.savannah.gnu.org:/sources/hurd co -r flaviocruz-soc2008-lisp-branch hurd +# mkdir hurd-build +# cd hurd-build +# ../hurd/configure --prefix=/usr --enable-lisp +# make +# make install +@end example - * (asdf:operate 'asdf:load-op 'tree-translator) +With everything in place I recommend you to previously compile everything. In the CLISP read-eval-print loop do: +@example +* (asdf:operate 'asdf:load-op 'tree-translator) +@end example -------------------------------- -4. A /dev/zero like translator -------------------------------- +@node A /dev/zero like translator, A more complex example, Top, Top +@chapter A /dev/zero like translator Now that we can really starting doing some interesting things, we will start with a basic translator that can easily introduce ourselves to important concepts and at the same time don't be overly complex. -The translator we will be implementing is very similar to the /dev/zero file that probably any Unix like system has. This file ignores everything that is written to it and when it is read it only gives us zeroes. It also very similar to the HHG's one.c example. +The translator we will be implementing is very similar to the @file{/dev/zero} file that probably any Unix like system has. This file ignores everything that is written to it and when it is read it only gives us zeroes. It also very similar to the HHG's one.c example. -Before the actual code we will properly package the new translator, using the ASDF method. Let's create a file named zero-translator.asd, with the following content: +Before the actual code we will properly package the new translator, using the ASDF method. Let's create a file named @file{zero-translator.asd}, with the following content: +@lisp (defpackage #:zero-translator-asd (:use :cl :asdf)) @@ -88,68 +160,128 @@ Before the actual code we will properly package the new translator, using the AS :description "/dev/zero translator." :depends-on (:hurd-translator) :components ((:file "zero"))) +@end lisp -As you can see, this ASDF package has only one file (zero.lisp) and only depends on the hurd-translator ASDF package, which is the basic translator library that we will be using. If you need to use more ASDF packages please declare them there. +As you can see, this ASDF package has only one file (@file{zero.lisp}) and only depends on the hurd-translator ASDF package, which is the basic translator library that we will be using. If you need to use more ASDF packages please declare them there. -Now to the zero.lisp file. At the top we put: +Now to the @file{zero.lisp} file. At the top we put: +@lisp (defpackage :zero-translator (:use :cl :hurd-common :mach :hurd :hurd-translator)) (in-package :zero-translator) +@end lisp + This basically tells the Lisp system to create a new package which uses the Hurd translator library (hurd-common, mach, hurd, hurd-translator and tree-translator packages are all from this library) and then sets the currently used package to the zero-translator package. Now inside the new package, we must define a new translator class, so that we can customize the translator methods to this new class. These translator methods act like hooks/callbacks that run whatever some request is made upon the server. +@lisp + (defclass zero-translator (translator) () (:documentation "The zero-translator.")) +@end lisp + This definition is remarkably simple, it just tells that the translator class is the base class of the zero-translator class and doesn't add any new slots. -If you want to see how the translator class is defined please look at the file translator/class.lisp. The important slots stored on this class follow: +If you want to see how the translator class is defined please look at the file @file{translator/class.lisp}. The important slots stored on this class follow: + +@itemize @bullet +@item +underlying-node + +It's the port to the underlying node where the translator is set. You can execute RPC's on it to request the information you need. For RPCs look at hurd/io, hurd/fs and hurd/fsys. + +@item +root + +It's simple the root node of the translator. If the translator simple exposes a new file this is that file, if it's a directory it will point to a directory node. + +@item +get-statfs - (1) underlying-node: It's the port to the underlying node where the translator is set. You can execute RPC's on it to request the information you need. For RPCs look at hurd/io, hurd/fs and hurd/fsys. - (2) root: It's simple the root node of the translator. If the translator simple exposes a new file this is that file, if it's a directory it will point to a directory node. - (3) get-statfs: Contains file system meta data that should be set by the translator programmer. Look at common/statfs.lisp for the methods you can use on this object. - (4) storage: Type of storage for this file system, defaults to :memory. Look at hurd/fs/storage.lisp for more types. - (5) options: Currently used file system options that can be set using the fsysopts utility. It's a translator-options object and it's implemented in hurd/translator-options.lisp. - (6) name and version: Translator name and 3 number list with version. It's used in request to the io-server-version RPC. +Contains file system meta data that should be set by the translator programmer. Look at @file{common/statfs.lisp} for the methods you can use on this object. -Now, you also must know what kinds of methods can be applied to a translator object, so that you can get the desired translator behavior. All these callbacks are extensively documented and described in the file translator/api.lisp along with some good default implementations. -If you want to dig at a lower level, it's also possible to define how core RPC's like fsys-getroot and dir-lookup work, specializing the do-fsys-getroot and do-dir-lookup (look at translator/interfaces/fsys-getroot.lisp and translator/interfaces/dir-lookup.lisp, respectively). +@item +storage -And if these methods don't satisfy you can even replace the lower level callbacks (implemented at translator/interfaces/), but be aware that you must deal with nasty things, like pointers, memory, etc and you must also be knowledgeable in using CFFI to deal with foreign calls. +Type of storage for this file system, defaults to :memory. Look at @file{hurd/fs/storage.lisp} for more types. + +@item +options + +Currently used file system options that can be set using the fsysopts utility. It's a translator-options object and it's implemented in @file{hurd/translator-options.lisp}. + +@item +name and version + +Translator name and 3 number list with version. It's used in request to the io-server-version RPC. + +@end itemize + +Now, you also must know what kinds of methods can be applied to a translator object, so that you can get the desired translator behavior. All these callbacks are extensively documented and described in the file @file{translator/api.lisp} along with some good default implementations. +If you want to dig at a lower level, it's also possible to define how core RPC's like fsys-getroot and dir-lookup work, specializing the do-fsys-getroot and do-dir-lookup (look at @file{translator/interfaces/fsys-getroot.lisp} and @file{translator/interfaces/dir-lookup.lisp}, respectively). + +And if these methods don't satisfy you can even replace the lower level callbacks (implemented at @file{translator/interfaces/}), but be aware that you must deal with nasty things, like pointers, memory, etc and you must also be knowledgeable in using CFFI to deal with foreign calls. All right, let's forget about that and get to the good stuff! -Now that you have some more understanding how everything fits together, we should start to implement our first translator method: make-root-node. +Now that you have some more understanding how everything fits together, we should begin implementing our first translator method: @code{make-root-node}. -This method is only called right at the beginning of execution and it wants us to create a new 'node' object that will represent the initial translator file (which can be anything: a regular file, directory, character device, etc). +This method is only called right at the beginning of execution and it wants us to create a new 'node' object that will represent the initial translator file, which can be anything: a regular file, directory, character device, etc. To define translator methods we use the define-callback macro, which has this syntax: -(define-callback <method-name> <translator-type> <argument list> <@body>) +@lisp +(define-callback <method-name> <translator-type> <argument list> <body>) +@end lisp Which is transformed (as you may be guessing) into this: -(defmethod <method-name> ((translator <translator-type>) @<argument list>) - <@body>) +@lisp +(defmethod <method-name> ((translator <translator-type>) <argument list>) + <body>) +@end lisp + +As you noted, the translator argument is always present in translator methods! It represents your yet to be created, translator object, of the class zero-translator. Apart from that 'hidden' argument, you can also use the *translator* dynamic variable that references the exact same object, but can be accessed anytime and anywhere you want. + +Before implementing the @code{make-root-node} callback, you first must know the node class. It's implemented in the file @file{translator/node.lisp} and has the following important data: + +@itemize @bullet +@item +owner + +File owner. Indicates the owner process ID and can be changed through the io-mod-owner RPC. Defaults to 0. -As you noted, the translator argument is always present in your methods! It represents your yet to be created, translator object, of the class zero-translator. Apart from that 'hidden' argument, you can also use the *translator* dynamic variable that references the exact same object, but can be accessed anytime and anywhere you want. +@item +stat -Before implementing the make-root-node callback, you first must know the node class. It's implemented in the file translator/node.lisp and has the following important data: +This is a very important slot. It represents the node meta data and it's the same as the C struct stat. You really should know how to retrieve and change the stat information. Please look at @file{common/stat.lisp} to have a basic idea. Also note that the stat meta data also contains a mode object, which itself, describes node permissions, file type, among other things, look at @file{common/mode.lisp} to see what methods are available to you. Important node: every method applicable to a mode object can also be directly applied to a stat object. - (1) owner: File owner. Indicates the owner process ID and can be changed through the io-mod-owner RPC. Defaults to 0. - (2) stat: This is a very important slot. It represents the node meta data and it's the same as the C struct stat. You really should know how to retrieve and change the stat information. Please look at common/stat.lisp to have a basic idea. Also note that the stat meta data also contains a mode object, which itself, describes node permissions, file type, among other things, look at common/mode.lisp to see what methods are available to you. Important node: every method applicable to a mode object can also be directly applied to a stat object. - (3) box: Translator box, can contain the current passive or/and active translator set on this node. - (4) nusers: Number of users using this node. When it drops to 0 the method report-no-users is called and when it gets from the 'no users' state to a new user, the method report-new-user is called. - (5) link: Contains a string indicating the file this node symbolic links to. Only makes sense when the node mode type is a link. +@item +box -Alright, here is the make-root-node method: +Translator box, can contain the current passive or/and active translator set on this node. +@item +nusers + +Number of users using this node. When it drops to 0 the method report-no-users is called and when it gets from the 'no users' state to a new user, the method report-new-user is called. + +@item +link + +Contains a string indicating the file this node symbolic links to. Only makes sense when the node mode type is a link. +@end itemize + +Alright, here is the @code{make-root-node} method: + +@lisp (define-callback make-root-node zero-translator (underlying-node underlying-stat) (declare (ignore underlying-node)) @@ -160,28 +292,50 @@ Alright, here is the make-root-node method: (make-instance 'node :stat (make-stat underlying-stat :mode mode)))) +@end lisp The first thing we must notice here are the arguments. This method accepts the underlying node port and 'underlying-stat', which refers to the stat meta data from the underlying node where the translator is being set. The first thing we do, is creating a new sane mode meta data, indicating the types of permissions we want for the node and then the file type, in this case character device :chr. Other possible types are: - (1) :dir - Directory. - (2) :chr - Character device. - (3) :blk - Block device. - (4) :reg - Regular file. - (5) :lnk - Symbolic link. - (6) :sock - Socket. +@itemize @bullet +@item +:dir + +Directory. +@item +:chr + +Character device. +@item +:blk + +Block device. +@item +:reg + +Regular file. +@item +:lnk + +Symbolic link. +@item +sock -With the mode object created, we then create a node instance, indicating the stat object we want for that node, with make-stat. +Socket. +@end itemize -make-stat accepts an old stat object, and so the meta data will be copied to the new stat object. It means that this new stat object will be equal to 'underlying-stat', except the mode object which will be the node we created early. make-stat can also accept other arguments, please see common/stat.lisp. +With the mode object created, we then create a node instance, indicating the stat object we want for that node, with @code{make-stat}. + +@code{make-stat} accepts an old stat object, and so the meta data will be copied to the new stat object. It means that this new stat object will be equal to 'underlying-stat', except the mode object which will be the node we created early. @code{make-stat} can also accept other arguments, please see @file{common/stat.lisp}. In the end, the newly created node is returned and put in the root-node translator slot, so that you can access it later on, if you need. What else needs to be implemented? :-) Ah, file writing! -File writing is pretty simple on this translator, we only need to ignore everything we get. The method that deals with file writing is write-file. +File writing is pretty simple on this translator, we only need to ignore everything we get. The method that deals with file writing is @code{write-file}. +@lisp (define-callback write-file zero-translator (node user offset stream amount) (declare (ignore translator offset amount)) @@ -189,24 +343,52 @@ File writing is pretty simple on this translator, we only need to ignore everyth ; Empty the stream to look like we used it all. (loop while (read-byte stream nil)) t)) +@end lisp + +Here, we can see that @code{write-file} gets lots of arguments, which I will explain: + +@itemize @bullet +@item +node + +It's the node the user wants to write in. +@item +user + +The user making the write request. It's an iouser object contain the uid and gid sets from that user. Implemented at @file{hurd/iohelp/iouser.lisp}. + +@item +offset -Here, we can see that write-file gets lots of arguments, which I will explain: +The place in the file the user wants to start writing to. - (1) node: It's the node the user wants to write in. - (2) user: The user making the write request. It's an iouser object contain the uid and gid sets from that user. Implemented at hurd/iohelp/iouser.lisp. - (3) offset: The place in the file the user wants to start writing to. - (4) stream: An input byte stream containing the data to be wrote, you can used read-byte to get the data, or read-sequence. - (5) amount: Amount of bytes present in this stream. Very useful if you want to put data into an array: +@item +stream + +An input byte stream containing the data to be wrote, you can used read-byte to get the data, or read-sequence. + +@item +amount + +Amount of bytes present in this stream. Very useful if you want to put data into an array: +@lisp (let ((my-array (make-array amount :element-type '(unsigned-byte 8)))) (read-sequence my-array stream)) +@end lisp + +You can even get it in string form if it makes sense: + +@lisp +(octets-to-string my-array) +@end lisp - You can even get the string if it makes sense: - (octets-to-string my-array) +@end itemize How can the library know how much has been wrote? Well, simple looking at how much you read from the stream. So, if you read everything from the stream, the library will get the idea that everything was used, and then we return T meaning success. -How about file reading? read-file is the method for reading nodes. +How about file reading? @code{read-file} is the method for reading nodes. +@lisp (define-callback read-file zero-translator (node user start amount stream) (declare (ignore translator start)) @@ -214,78 +396,107 @@ How about file reading? read-file is the method for reading nodes. (loop for i from 0 below amount do (write-byte 0 stream)) t)) +@end lisp The arguments, explained: - (1) node: The node where the user wants to read. - (2) user: The user making the request. - (3) start: The file position the user wants to start reading at. - (4) amount: Amount of data the user wants to read. You don't need to provide that amount of data, only what you have or want deliberately. - (5) stream: Output stream were byte data should be written to. +@itemize @bullet +@item +node + +The node where the user wants to read. + +@item +user + +The user making the request. + +@item +start + +The file position the user wants to start reading at. + +@item +amount + +Amount of data the user wants to read. You don't need to provide that amount of data, only what you have or want deliberately. + +@item +stream + +Output stream were byte data should be written to. + +@end itemize As we said earlier, the translator should give back only zeroes to read requests, because of that we simply write 'amount' zeroes requested to the output stream, and then return T signaling success. What's missing? Ah! Start-up code: +@lisp (defun main () (run-translator (make-instance 'zero-translator :name "zero-translator"))) (main) +@end lisp Here we just create a new zero-translator instance and then run it. Boring, isn't it? -And that's it! Let's wrap everything with a new file: run-zero.lisp: +And that's it! Let's wrap everything with a new file: @file{run-zero.lisp}: +@example #!/usr/bin/run-lisp-trans (asdf:operate 'asdf:load-op 'zero-translator) +@end example Now we can run the translator: +@example $ chmod +x run-zero.lisp + $ settrans -ac zero ./run-zero.lisp +@end example And then you verify that it's really a character device: +@example $ ls -l zero crw-rw-rw- 2 root root 0, 0 Aug 14 15:09 zero +@end example You can also do some tests. Run a CLISP instance and do this: +@example * (asdf:operate 'asdf:load-op 'hurd) - * (use-package :hurd) - * (defvar *p* (file-name-lookup "zero" :flags '(:read :write))) - * (io-read *p* :amount 5) #(0 0 0 0 0) - * (io-write *p* #(1 2)) 2 - * (use-package :mach) - * (port-deallocate *p*) T +@end example As expected. Remove the translator: +@example $ settrans -g zero +@end example - -------------------------------- - 5. A more complex example. -------------------------------- +@node A more complex example, Conclusion, Top, Top +@chapter A more complex example Now we'll trying to implement a more complex translator example. The translator we will implement will provide directory listing and the usual file creation, writing, etc. At first it should create a directory structure mirroring Lisp lists. Here's an example: +@lisp (:dir "root" (:file "a" "abcdefghijklmnopqrstuvwxyz") (:file "b" "123456789012345678901234567890") @@ -301,13 +512,15 @@ At first it should create a directory structure mirroring Lisp lists. Here's an (:dir "dir3" (:file "a" "12345")) (:link "g" "g") ; circular link - (:link "h" "/usr") - ) + (:link "h" "/usr")) -Our translator node must have a directory named "root" and then some files and links on it... and so on. +@end lisp + +Our translator node must have a directory named "root" and then some files and links on it. The third list argument for regular files describe file contents and the third argument for symbolic links describe the link target. First, let's describe this new translator class (we'll name it test-translator): +@lisp (defclass test-translator (tree-translator) ((file-stat :initarg :file-stat :initform nil @@ -315,23 +528,25 @@ First, let's describe this new translator class (we'll name it test-translator): (dir-stat :initarg :dir-stat :initform nil :accessor dir-stat))) +@end lisp -You might be wondering what's up with that 'tree-translator' business. We'll, the tree-translator is a special class of translators that do some work for us. The tree-translator has the special ability of implementing all the directory lookup and listing automatically, without the need for us to do that. If you want to see what methods are implemented please see tree-translator/class.lisp. +You might be wondering what's up with that 'tree-translator' business. We'll, the tree-translator is a special class of translators that do some work for us. The tree-translator has the special ability of implementing all the directory lookup and listing automatically, without the need for us to do that. If you want to see what methods are implemented please see @file{tree-translator/class.lisp}. It also extends the base node class with two new node types: * entry: Is a node with the information about the parent directory. * dir-entry: Is an 'entry' and also contains the directory entries. -Please check the file tree-translator/dir.lisp for methods on directories. +Please check the file @file{tree-translator/dir.lisp} for methods on directories. Some important meta data is also managed by this type of translator, specially the st-nlink and st-ino fields from the stat object. In our test-translator we put the file-stat and dir-stat slots. They act like a template meta data for new nodes. -Another thing you should take note is that the previously used method make-root-node is already implemented for us in tree-translator's, in addition it creates a new method named fill-root-node with the root node as an argument. In this method you should fill your root directory with the stuff you want. +Another thing you should take note is that the previously used method @code{make-root-node} is already implemented for us in tree-translator's, in addition it creates a new method named @code{fill-root-node} with the root node as an argument. In this method you should fill your root directory with the stuff you want. For this case, this might be implemented like this: +@lisp (define-callback fill-root-node test-translator ((node dir-entry)) (setf (file-stat translator) @@ -347,9 +562,11 @@ For this case, this might be implemented like this: (%fill-node translator (with-open-file (s +file+) (read stream)) node)) +@end lisp -As you can see it defines the translator slots with fresh meta data templates and then executes the function %fill-node with arguments the translator, the directory structure (as read from the +file+) and the root node. +As you can see it defines the translator slots with fresh meta data templates and then executes the function @code{%fill-node} with arguments the translator, the directory structure (as read from the @code{+file+}) and the root node. +@lisp (defun %fill-node (translator ls node) (let ((type (first ls)) (name (second ls)) @@ -380,42 +597,50 @@ As you can see it defines the translator slots with fresh meta data templates an :parent node))) (setf (link new) target) (add-entry node new name)))))) +@end lisp -This is a recursive function. It checks the wanted file type, creates the specific node with that type and then it is put in the directory as a new directory entry. +This is a recursive function. It checks the requested file type, creates the specific node with that type and then it is put in the directory as a new directory entry. You should always give the :parent argument when you are creating new 'entry's. -Take special attention for the symbolic links case "(setf (link new) target)". It defines the link (as a string) this link points to, but it also sets the file type to :lnk, even if it was not specified above. +Take special attention for the symbolic links case "@code{(setf (link new) target)}". It defines the link (as a string) this link points to, but it also sets the file type to :lnk, even if it was not specified above. You may also be wondering what's that test-entry thing, but, it's just a new kind of node/entry. +@lisp (defclass test-entry (entry) ((contents :initarg :data :initform (%create-data-array 0 nil) :accessor data))) +@end lisp In this case we add a new slot that contains the file data, and that %create-data-array just creates a new adjustable array: +@lisp (defun %create-data-array (size contents) (make-array size :initial-contents contents :adjustable t :fill-pointer t :element-type '(unsigned-byte 8))) +@end lisp You can use the CLOS dispatch system to specialize node behaviors, for example, you can have various node types that react differently to the read-file method, one would always return the data in upper case and in some others all in lower case, for example. -As for that %read-file-data function, here it is: +As for that @code{%read-file-data} function, here it is: +@lisp (defun %read-file-data (str) (%create-data-array (length str) (loop for char across str collect (char-code char)))) +@end lisp It takes the data string and converts it to an unsigned-byte array. -Now that the directory structure is being created, you need to start exposing the file contents with read-file (just like the first case). Please also keep in mind that you should always keep the st-size stat field updated after a refresh-node method call. Sometimes the translator clients want to know how much data there is to read, and the file size is an important factor in this case, when it's being compared to an internally saved file offset. +Now that the directory structure is being created, you need to start exposing the file contents with @code{read-file} (just like the first case). Please also keep in mind that you should always keep the st-size stat field updated after a refresh-node method call. Sometimes the translator clients want to know how much data there is to read, and the file size is an important factor in this case, when it's being compared to an internally saved file offset. +@lisp (define-callback read-file test-translator (node user start amount stream) (when (has-access-p node user :read) @@ -428,11 +653,13 @@ Now that the directory structure is being created, you need to start exposing th (write-sequence (subseq (data node) start end) stream) t)))) +@end lisp -In the read-file method we check if the user has read access to that node and then calculate how much data there is left to read from the 'start' position. When we know that, we just write a sub sequence of the file contents to the output stream and then return T to report success. +In the @code{read-file} method we check if the user has read access to that node and then calculate how much data there is left to read from the 'start' position. When we know that, we just write a sub sequence of the file contents to the output stream and then return T to report success. -Another good thing to provide in translators (specially those with write support) is file truncating. For that, the library has the file-change-size method. +Another good thing to provide in translators (specially those with write support) is file truncating. For that, the library has the @code{file-change-size} method. +@lisp (define-callback file-change-size test-translator (node user new-size) (when (is-dir-p (stat node)) @@ -441,11 +668,13 @@ Another good thing to provide in translators (specially those with write support (adjust-array (data node) new-size :fill-pointer t) (setf (stat-get (stat node) 'st-size) new-size) t)) +@end lisp First we verify if the file is a directory, and then if the requesting user has write access. When these tests are passed we can just adjust the file contents array and define a new size for that file in the stat object. And now for file writing! +@lisp (defun %read-sequence (stream amount) (let ((arr (make-array amount :element-type '(unsigned-byte 8)))) @@ -469,15 +698,17 @@ And now for file writing! ; Update stat size. (setf (stat-get (stat node) 'st-size) final-size) t)) +@end lisp -Well, let's see. We first do some boring tests and then we read all data from the stream with %read-sequence, then we need to know how much data the file will now have, and there three possible situations: the data will just be replaced, the file will get new data and both. Anyway, we need just to know the maximum of the current size and the offset plus the new data. +Well, let's see. We first do some boring tests and then we read all data from the stream with @code{%read-sequence}, then we need to know how much data the file will now have, and then we have three possible situations: the data will just be replaced, the file will get new data and both. Anyway, we just need to know the maximum of the current size and the offset plus the new data. After knowing the final size and if the file will grow we adjust the array according to this new size. Then simply replace the current array data with the stream data and update the file size. It's that simple. -Now, you'll probably want to support file creation, for that, you must implement the create-file method. +Now, you'll probably want to support file creation, for that, you must implement the @code{create-file} method. +@lisp (define-callback create-file test-translator (node user filename mode) (unless (has-access-p node user :write) @@ -489,13 +720,29 @@ Now, you'll probably want to support file creation, for that, you must implement :parent node))) (add-entry node entry filename) entry)) +@end lisp The create-file method has 4 arguments: - (1) node: The directory where the user wants to create the file. - (2) user: The user creating the file. - (3) filename: The new file name. - (4) mode: Permission bits for that file. +@itemize @bullet +@item +node + +The directory where the user wants to create the file. + +@item +user + +The user creating the file. +@item +filename + +The new file name. +@item +mode + +Permission bits for that file. +@end itemize As you can see, we just create a new test-entry instance with 'node' has the parent and the you add that entry to the directory and return it. @@ -503,12 +750,26 @@ How about hard link creation? It's already implemented by default in the tree-tr And symbolic link creation? Two methods must be implemented for it to work: - (1) allow-link-p: Allows link reading, implemented by default by the translator class. - (2) create-symlink: Turns a node into a symlink, implemented by default by the translator class. - (3) create-anonymous-file: Not implemented. It must create a detached node. Detached means that file will have a parent directory but not present when listed, and it will exist as a Lisp object that will be hard linked to a new symlink node. See sysdeps/mach/hurd/symlink.c from glibc's source code for more insights and reasons for this. +@itemize @bullet +@item +@code{allow-link-p} + +Allows link reading, implemented by default by the translator class. + +@item +@code{create-symlink} + +Turns a node into a symlink, implemented by default by the translator class. + +@item +@code{create-anonymous-file} + +Not implemented. It must create a detached node. Detached means that file will have a parent directory but not present when listed, and it will exist as a Lisp object that will be hard linked to a new symlink node. See sysdeps/mach/hurd/symlink.c from glibc's source code for more insights and reasons for this. +@end itemize Here's a simple implementation for create-anonymous-file: +@lisp (define-callback create-anonymous-file test-translator (node user mode) (when (can-modify-dir-p node user) @@ -516,17 +777,22 @@ Here's a simple implementation for create-anonymous-file: :stat (make-stat (stat node) :mode mode) :parent node))) +@end lisp + Here 'node' represents the parent directory and 'mode' the permission bits. How about the start-up code? First we need to get the file from the arguments that will be passed to our program. This file contains the mentioned directory structure. +@lisp (unless (= (length ext:*args*) 1) (error "You must provide one argument with a file.")) (defconstant +file+ (first ext:*args*)) +@end lisp Then the usual main function: +@lisp (defun main () (let ((translator (make-instance 'test-translator @@ -535,11 +801,13 @@ Then the usual main function: (run-translator translator))) (main) +@end lisp -Please see examples/test.lisp, examples/test-translator.asd and examples/run-test.lisp for the complete example. +Please see @file{examples/test.lisp}, @file{examples/test-translator.asd} and @file{examples/run-test.lisp} for the complete example. To see it running: +@example $ settrans -ac foo ./run-test.lisp data/test.lisp $ cd foo $ ls @@ -557,10 +825,10 @@ dr-xr-x--- 3 root root 0 Aug 14 19:13 dir3 lr--r----- 1 root root 1 Aug 14 19:13 f -> c lr--r----- 1 root root 1 Aug 14 19:13 g -> g lr--r----- 1 root root 4 Aug 14 19:13 h -> /usr +@end example -------------------------------- - 6. Conclusion -------------------------------- +@node Conclusion, Top, Top +@chapter Conclusion Now that you got some taste of what's like to program Lisp translators, I invite you to read more complex translator examples that are present in the package, namely the zip and irc translators. |