Home   Info   DevZone   Wiki   UsersWeb  |  MainWeb  |  InfoWeb  |  DevZoneWeb  |  SupportWeb
[ FilesystemDiscussion2 ] [ not logged in ] [ Web: Imported ] goto:  options [ get info on or edit ] [ login or new user ] [ list of topics, hubs & nodes, or recent changes ]

Up: FilesystemGroup
Prev: FilesystemDiscussion

The information on this page may be outdated; it reflects the discussion led/being led about the subject "Filesystem for JOS". Feel free to join!

DavidChristie sent me an excellent email that might provide a good starting point. David pointed out to me that two crucial concepts underlying organizational schemes like directory based file systems are Identity and Aggregation. For example, a traditional file system identifies files in terms of the file's name, date, and extension, and Aggregation for files seems to be simply a hierarchical based relationship. With XML and object oriented methodologies, these 'files' can have a much richer identity and aggregation based on XML meta-data, XML extended links, raw file location, abstract URL location, class name, date of creation, authoring tool, etc. Can we create an interface/interfaces around the concept of Identity and Aggregation that could allow all three of us to pursue our own file system concepts while retaining some sort of interoperability? Put another way, is there a DesignPattern that underlies all of our ideas that might provide a foundation for each of us?

This is a challenge that needs a great designer! If you're out there, help us! May the GangOfFour bless our enterprise..... ahm..... now the holy chant.... reuse is good... do not inherit, my child, aggregate. Amen.

BradNeuberg, bkn3@columbia.edu

Brad, :-)!!

Your comments on Identity and Aggregation could prove very useful, indeed. I'm still in the process of thinking about an adequate FileSystem interface design. These could be approaches:

• String based
  Example: fileSystem.delete("/home/misc/abc.txt");

• Directory tree based (dumb nodes)
  Example:

  FSINode node = fileSystem.getRootDir();
  node = fileSystem.getElement(node, "home");
  node = fileSystem.getElement(node, "misc");
  node = fileSystem.getElement(node, "abc.txt");
  fileSystem.delete(node);
  

• Directory tree based (intelligent nodes)
  Example: fileSystem.getRootDir().getElement("home").getElement("misc").getElement("abc.txt").delete();

• Identity/Aggregation based
  Example: ?

We'll probably need more than one interface, with bridges between them.

--StefanReich, 02-13-98

in addition, we have to decide if we want/how to retain file compatibility with java.io's stuff and traditional file/directory systems

IMO, we don't really have a choice. We need java.io to have existing Java apps run under JOS, and we need access to (not necessarily compatibility with) traditional FS to allow data exchange between JOS and the rest of the world.

--StefanReich, 02-13-98

// Get the starting point, or the home, of this person's web
PersonalWeb home = web.getHome();

// getElement takes an static Identity (could be a String as well)
// and  the name of a node under the given Identity.
node = home.getElement(Identity.FILENAME, "essay1.txt");
node = home.getElement(Identity.FILENAME, "misc");
node = home.getElement(Identity.BUILT_WITH, "mpEdit", "resume.txt");
node = home.getElement(Identity.BUILT_WITH, "Find", "preferences");

// three different possibilities for handling directories
// possibility 1
Enumeration directory = (Enumeration)home.getElementEnumeration(Identity.BUILT_WITH, "mpEdit");
directory.setDirectoryName("Files Created with mpEdit");

while (directory.hasMoreElements()) {
	node = directory.nextElement();
	display(node);
}

// possibility 2
Directory directory = home.getElements(Identity.BUILT_WITH, "mpEdit");
TraditionalFileSystemView filesystem = new TraditionalFileSystemView();
filesystem.display(directory);

//possibility 3; the concept of directories is expanded to include nodes
// that provide navigational assitance to the user
DirectoryNode directory = home.getDirectory(Identity.BUILT_WITH, "mpEdit");
Win95View directory_view = new Win95View();
directory_view.directory();

// I'm still really hazy on aggregation, so I'm not sure if
// this is the correct way to do it.  This is a Model-View-Controller
// aggregation that 'parades' as a single object; using the
// terminology of Linus, I'll call it a DocSet object.
// It takes a Semantic file (XML), a Rendering file (Cascading Style
// Sheet), and a Control object (Java class)
DocSetObject document = new DocSetObject("email1.xml", "MemoView.css", "EmailNode.class");

document.start();
// maybe you could now manipulate the aggregate docset object using
// the Document Object Model (DOM) or Linus's DocSet tools, or maybe both?

// pseudo-DOM
document.body.div.display.color.setColor("blue");

// pseudo-DocSet
DocSetTools docTools = getDocSetToolkit();
docTools.select(document);
docTools.delete(document);
docTools.dragdrop(document1, document2); // dragdrop(from, to)
docTools.setBackground(document, "blue");
try { docTools.blur(document); } catch(OperationNotSupported e) {}
docTools.email(document);
docTools.wikify(document);
docTools.placeonpage("MyHomePage", "embed", document);
docTools.placeonpage("JosMainPage", "link", document);
// morph a document from Semantic type 'webpage' to Semantic
// type 'email'
document = docTools.morph(document, "webpage.dtd", "email.dtd");  
// Semantically, the 'Author' of a webpage document is the 'From'
// of an email document
document.setFrom("bkn3@columbia.edu");
document = docTools.morph(document, "email.dtd", "webpage.dtd");
String author = document.getAuthor();
System.out.println(author);
// will print out "bkn3@columbia.edu"

I really believe in having working code upfront. I think Stefan's model is a good place to start. I would start coding it, but I'm working on the XML JavaBean nodes for the next few weeks, which might come in handy down the road.

-- BradNeuberg, 2/13/98

^..^ May you have a good.... Friday the 13th.... hahahahehehe... just kidding.

Brad, looks like we can form a good team: you think as an application programmer (top-down), I think as a system programmer (bottom-up).

The identity/aggregation concept is very interesting, but I don't think I really got the idea. For example, what exactly does this line mean:

node = home.getElement(Identity.BUILT_WITH, "mpEdit", "resume.txt");

Does it say "get a file named resume.txt that was built with mpEdit?"

Anyway, the next thing I'm going to do is to start from the bottom and finish a working implementation of a FAT (forgive me, folks) file system, with both read and write access. This shouldn't take too long.

Then I can proceed upwards, check back with you and see how we can connect our approaches.

--StefanReich, 2/13/98

(Friday, 13th? Oh, that's why everything goes wrong today...)

Good idea to finish the FAT system and the work you've done so far; it's always best to have working code. I'm going to continue to implement the XML stuff I'm doing. Maybe in about a month's time we will both have enough experience, you from the bottom up, me from the top down, to understand the issues involved in setting up the system we've been talking about. I'm only now getting a feel for the limitations and possibilities of XML, and you are probably gaining the same awareness of the low-level file system. You're interpretation of home.getElement is correct. Every web object might have an appropriate authoring object associated with it, so the getElement command above would create a nice directory with every file that was authored with mpEdit.... I guessed I goofed by also putting a filename there.

--BradNeuberg, 2-13-98

Hi, I have a couple of questions. First, why only have one file system? I know this sounds silly but, if you have a low level "block" finder that actually accesses the devices. You can then layer n-number file systems on top. For example I designed an object based file system, that also accounted for objects aswell. There where two methods for accessing the files/objects. One, was thought a message call (the OS was very message connected - slow but safe) to the object file system, and the second was to the "file" system. I also allowed for complex file types that applications could be "packaged" with the source code. Each part of the file-object had its own unix alike file permissions.

Also, I allowed for a global file system that all objects belonged things called "realms" so that the file-systems would go and find where the objects actually existed. So, that any application did not care where the actual object lived.

Hi PeterAntoine (I guess it was you wrote the comment above),

that's a lot of good ideas you mention. But you have to remember one thing: we want to retain compatibility with the java.io.* classes, so existing applications run unchanged on JOS. We don't want to define new standards, we want to use existing ones. (These could also include JNDI, JNFS, etc.).

As I described in FilesystemArchitecture, my current proposal includes a Unix-like mount mechanism. This way, you can access multiple file systems by symbolic names without knowing where a particular object resides.

I also would like to include arbitrary symbolic links, but this leads to a problem: following symbolic links is easy, e.g.: new FileInputStream("/home/symlink/afile"), but defining symbolic links is not a feature of java.io.*. Once again, the "no new API" conflict.

--StefanReich, 2/18/98

At some point, no matter what logical structure your filesystem supports, it always comes down to writing bytes on some physical device. Whether you view the structure as a hierarchical directory tree or a persistent object store, it will ultimately have to be represented as tracks and sectors.

Most OSes do not make any distinction between the file-system-as-storage and the file-system-as-logical-structure. One notable example would be Amoeba, which actually had a separate process providing the file and directory structure on top of formatted storage. This might be an instructive example.

-MichaelNygard 2/26/1998

Tomorrow, I will set up a page InterfaceDefinitions where I describe these interfaces in greater detail (and invite other developers to provide information about their interfaces as well).

--StefanReich, 2/26/1998

--DigiGod

Ahmmm... what's KOH? Please explain.

--StefanReich, 16-MAR-98

any attempts to build a FS basing on balanced-tree like reiserfs ?
http://idiom.com/~beverly/reiserfs.html
I'm not informed about the copyright situation - is it legal to reimplement a filesystem code when I've seen the sources of reiserfs? I think the knowledge, how reiserfs is implementes in linux, forbids me to reimplement it in java. i don't know.. maybe reiser even helps jos implement reiserfs - he seems to be open minded concering free-operating systems (like linux).

btw: where should access control lists be stored in ext2fs or FAT?

hmm - what about building a "I/O Handler" for every file - a thing that supplies the cooked data instead of the standard raw data.. for example if you put a file call "httpd.pid" somewhere on the filesystem and ...

httpd.pid

#!/bin/josshell (thats just an example! please no /bin,/usr/,/var styled organisation)
echo "Content-type: mime/text/plain"
/sbin/pidof httpd

--TherP, 16-MAR-98

--DigiGod 3/17/98

these features should be implemented in the "lowlevel" block-device-drivers. for each transfer the block has to pass a chain of transformation methods - i've no encryption is active the first entry of the chain points directly to the device-driver. if not, it point f.e. to a IDEA transformation module, which fetches it's data from the next entry in the list, that could be a stegography module, a multiple device driver (raid0 drivers.. maybe), a network block device driver which fetches it's data from the net and supplies them to the IDEA module..) or whatever.. the simplest case is of course the direct link to a local disc (not a FS - a raw device).

hmm btw: we should use the date format in GoodStyle ;) every entry has different date-format :)

--TherP, 17-MAR-98

--BrillPappin 16-JAN-1999

Content of these pages are owned and copyrighted by the poster.
Hosted by: