←Older revision Revision as of 23:03, 8 May 2008 (10 intermediate revisions not shown.)Line 1: Line 1: = WordPress SIOC Import Plugin = = WordPress SIOC Import Plugin = -<onlyinclude>Enables to create blog posts based on the SIOC data from weblogs, mailing lists and other sources. +<onlyinclude>This plugin enables you to create blog posts in Wordpress by taking SIOC data from other weblogs (but also other sources like mailing lists, CMS, etc. are possible). -It demonstrates that you don't have to be a developer to benefit from the web of RDF data - any blog user can have some use for it.</onlyinclude>+It demonstrates that you don't have to be a developer to benefit from the web of RDF data - any blog user can have use for it.</onlyinclude> == Installation == == Installation == Line 8: Line 8: http://sparql.captsolo.net/2008/05/sioc_import_plugin.zip http://sparql.captsolo.net/2008/05/sioc_import_plugin.zip -# Download [http://sparql.captsolo.net/2007/05/sioc_wordpress_import.zip sioc_wordpress_import.zip], place it into the WordPress plugins directory: WORDPRESS_ROOT/wp-content/plugins/ +# Download [http://sparql.captsolo.net/2008/05/sioc_import_plugin.zip sioc_import_plugin.zip], place it into the WordPress plugins directory: WORDPRESS_ROOT/wp-content/plugins/ # Unzip the archive - a new directory 'sioc_php_import' should appear in the plugins directory # Unzip the archive - a new directory 'sioc_php_import' should appear in the plugins directory # Choose "Plugins" tab in the WordPress admin interface, find a plugin called "SIOC Import Plugin" and hit "Activate" to enable it. # Choose "Plugins" tab in the WordPress admin interface, find a plugin called "SIOC Import Plugin" and hit "Activate" to enable it. Line 20: Line 20: [[Image:Sioc-wordpress-import.png]] [[Image:Sioc-wordpress-import.png]] -Enter the URL of SIOC data and press "Process URL". The plugin will parse data and create posts. It will display some RDF/XML followed by "Post created. <Edit link>" if a post was successfully created. New posts will be published on the blog with the current date/time.+On the administrator's panel go to Options - SIOC Import.  +Enter the URL of the SIOC data of an individual blog post and press "Process URL". The plugin will parse data and create posts. It will display some RDF/XML followed by "Post created. <Edit link>" if a post was successfully created. New posts will be published on the blog with the current date/time. If there are comments to this post they will be imported, too. -Note: only posts with a title and content will be created. Sometimes SIOC data contain references to linked data elsewhere (e.g., a blog post linking to its comments) w/o including these details in the current files. Such resources will not have posts created for them. This is normal behaviour.+== Where to get SIOC data? == -It is best to use a SIOC RDF URL describing an individual post (which can be found by going to a blog post and following it's SIOC RDF autodiscovery link). You could use site's main SIOC profile, but that often contains only references to SIOC posts, but not their content.+=== Example data to test the importer === -== Where to get SIOC data? ==+Copy these URLs to import data into your Wordpress blog:  +   +* Example blog post with comments from [http://torrez.us/index.php?sioc_type=post&sioc_id=562 torrez.us] (RDF/XML created by Wordpress SIOC Exporter)  +* A post in RDFa from [http://www.w3.org/2007/08/pyRdfa/extract?uri=http://www.wikier.org/blog/the-heilmeier-catechism#post wikier.org] and [http://www.w3.org/2007/08/pyRdfa/extract?uri=http://www.wikier.org/blog/talk-in-deri#post another one] with an image (from Textpattern blog engine).  +   +=== Exporting your own data as SIOC === SIOC data can be generated from a number of different blog and social media sites (e.g., from your weblog) using SIOC plugins: SIOC data can be generated from a number of different blog and social media sites (e.g., from your weblog) using SIOC plugins:  +  +* [http://sioc-project.org/wordpress WordPress SIOC Export plugin] * [http://sioc-project.org/drupal Drupal SIOC module] * [http://sioc-project.org/drupal Drupal SIOC module] * [http://sioc-project.org/mailinglists SWAML - SIOC exporter for mailing lists] * [http://sioc-project.org/mailinglists SWAML - SIOC exporter for mailing lists] -* [http://sioc-project.org/wordpress WordPress SIOC plugin]+* [[http://wiki.sioc-project.org/index.php/Implementations#Exporters ... look here for more exporter plugins ...]] -* [http://sioc-project.org/exporters ... look here for more plugins ...]+ Anyone can generate SIOC data which act as a "computer-readable" mirror of your blog and CMS posts. Anyone can generate SIOC data which act as a "computer-readable" mirror of your blog and CMS posts. Line 43: Line 50: Tools this software uses: Tools this software uses: -* [http://arc.web-semantics.org/ ARC RDF framework for PHP] by Bengee+* [http://www4.wiwiss.fu-berlin.de/bizer/rdfapi/ RAP - RDF API for PHP] -* [http://sioc-project.org/phpapi SIOC PHP API]+ Feedback: captsolo at gmail Feedback: captsolo at gmail Line 59: Line 65: == SVN access == == SVN access == -A somewhat stable version using ARC:+The experimental, more advanced version 0.2 using RAP:  +   + svn co http://sw.deri.org/svn/uldis/2008/04/sioc_php_import_experimental  +   +The previous version 0.1 using ARC: svn co http://sw.deri.org/svn/uldis/2007/05/sioc_php_import/ svn co http://sw.deri.org/svn/uldis/2007/05/sioc_php_import/ -An experimental, more advanced version using RAP:  - svn co http://sw.deri.org/svn/uldis/2008/04/sioc_php_import_experimental  = ToDos = = ToDos =

Without content negotiation - - minor spelling fixes

←Older revision Revision as of 12:31, 7 May 2008 Line 27: Line 27: Appending or prepending "sioc" doesn't matter, just make a new URI Appending or prepending "sioc" doesn't matter, just make a new URI -(using slash vs. using hash: if it is a document it has to be a differnt document, so it must be slash - if you just append a hash and you ask for it, then you will still get the original html page)+(using slash vs. using hash: if it is a document it has to be a different document, so it must be slash - if you just append a hash and you ask for it, then you will still get the original html page) ''uldis:'' but we are adding another level of indirection. ''uldis:'' but we are adding another level of indirection. Line 34: Line 34: Depending on the con type you ask for you get one of those. Depending on the con type you ask for you get one of those. -dabri quotes ''timbl'': "usually the rdf version of something is too reduced and too fragmented - too lossy to count as the same thing" +danbri quotes ''timbl'': "usually the rdf version of something is too reduced and too fragmented - too lossy to count as the same thing" e.g. the foaf spec vs. the schema at the same URI - those are two separate things e.g. the foaf spec vs. the schema at the same URI - those are two separate things -''cygri:'' true when there is clearly more information, but with sioc post: all the content is there in the rdf (dabri: "just missing the banner ads"), only dressed up less, layout...+''cygri:'' true when there is clearly more information, but with sioc post: all the content is there in the rdf (danbri: "just missing the banner ads"), only dressed up less, layout... -I think for sioc:Post it's apporpriate to say html and rdf are the same thing - just html and rdf version of the same thing. <br/>+I think for sioc:Post it's appropriate to say html and rdf are the same thing - just html and rdf version of the same thing. <br/> ''danbri:'' In a couple of years it will be RDFa ''danbri:'' In a couple of years it will be RDFa

←Older revision Revision as of 09:37, 6 May 2008 (One intermediate revision not shown.)Line 23: Line 23: == Miscellaneous == == Miscellaneous == -Axel wants to revive the Expert Finder Initiative - do a bit more than is done in foaf and sioc towards combining different vocabs to model expertise and extract expertise from SIOC or FOAF data - from connections between people. initiative founded 2 years ago, always fell asleep again.+Axel wants to revive the Expert Finder Initiative - do a bit more than is done in foaf and sioc towards combining different vocabs to model expertise and extract expertise from SIOC or FOAF data - from connections between people. Axel has some technical interest in exporters. Some of his group's work might be helpful: they have some extensions of SPARQL and combinations of SPARQL with XQuery ([http://polleres.net/xsparql XSPARQL]) that could be useful to write wrappers. Axel has some technical interest in exporters. Some of his group's work might be helpful: they have some extensions of SPARQL and combinations of SPARQL with XQuery ([http://polleres.net/xsparql XSPARQL]) that could be useful to write wrappers. If someone writes wrappers which use XSLT and SPARQL then you might want to have a look at [http://polleres.net/xsparql XSPARQL]. If someone writes wrappers which use XSLT and SPARQL then you might want to have a look at [http://polleres.net/xsparql XSPARQL]. -He also discussed with Dan some rule extensions of rdf (which sparql uses as rules) to model implicit data - that could also be of interest to the SIOC project.+He also discussed with Dan some rule extensions of RDF (which sparql uses as rules) to model implicit data - that could also be of interest to the SIOC project. John gave an overview of recent activities in regards to data portability [http://www.johnbreslin.com/blog/2008/04/11/dataportability-lunch-meetup-in-london-opensocial-hackathon/], [http://www.johnbreslin.com/blog/2008/04/01/kingsley-remixes-my-dataportability-slides-as-data-accessibility-and-me-introducing-sioc-foaf-and-the-linked-data-web/] and the SIOC-o-sphere [http://www.johnbreslin.com/blog/2008/04/05/tales-from-the-sioc-o-sphere-7/]. John gave an overview of recent activities in regards to data portability [http://www.johnbreslin.com/blog/2008/04/11/dataportability-lunch-meetup-in-london-opensocial-hackathon/], [http://www.johnbreslin.com/blog/2008/04/01/kingsley-remixes-my-dataportability-slides-as-data-accessibility-and-me-introducing-sioc-foaf-and-the-linked-data-web/] and the SIOC-o-sphere [http://www.johnbreslin.com/blog/2008/04/05/tales-from-the-sioc-o-sphere-7/]. Line 38: Line 38: === Importing vs. aggregating === === Importing vs. aggregating === -Richard doesn't really see a big picture use case for importers. He thinks SIOC is more useful when you think in terms of aggregation instead of importing data and doubts that copying data from one source to the other will be needed very often. Maybe making a reference to where the data resides is more useful. Keep the data it at its original place and just fetch it dynamically, so you see the data it in your system like as if it was native (it is only an implementation issue why many applications actually cache the data). +Richard doesn't really see a big picture use case for importers. He thinks SIOC is more useful when you think in terms of aggregation instead of importing data and has doubts that copying data from one source to the other will be needed very often. Maybe making a reference to where the data resides is more useful. Keep the data at its original place and just fetch it dynamically, so you see the data it in your system as if it was native (it is only an implementation issue why many applications actually cache the data). -Richard: "Maybe it would be useful to have something like this: I add a sioc subscription to my blog and whaterver post are on that other sioc source shows up seemlessly in my blog - I don't care if it is cached to fetched dynamically, it just should shows up on my blog"+Richard: "Maybe it would be useful to have something like this: I add a sioc subscription to my blog and whaterver post are on that other sioc source shows up seamlessly in my blog - I don't care if it is cached or fetched dynamically, it just should shows up on my blog" Danh says this is also what the Joomla users he works with want: they are not ready to export their data - but they are willing to have a hot link to their stuff from other sites. They want their data ready for mashup, but they don't want to lose control. Danh says this is also what the Joomla users he works with want: they are not ready to export their data - but they are willing to have a hot link to their stuff from other sites. They want their data ready for mashup, but they don't want to lose control. -Richard: could you do a import similar to Wordpress SIOC Importer also with the Atom publishing protocol? Because that is widely supported, it could be used to get imports with less effort.+Richard: could you do a import similar to Wordpress SIOC Importer also with the Atom publishing protocol? Because that is widely supported, it could be used to make importers with less effort. Sergio: you don't get everything, it is harder to get the comments or other parts of the data (categories...)<br> Sergio: you don't get everything, it is harder to get the comments or other parts of the data (categories...)<br> -> we don't know if you can get all of this published with the atom protocol -> we don't know if you can get all of this published with the atom protocol Line 51: Line 51: Where can we go from here with our importer? Where can we go from here with our importer? -Uldis: One of the next things we can do is: take a topic or category hierachy from the original post and re-create them for this target post - or in fact we shouldn't only be thinking in terms of post, there are other kind of objects, we could even just export SKOS categories and automatically create a category tree on another blog. Maybe we don't even need to import it - but just get the category hierachy from the source blog and display a user interface that says: "this is the category hierachy, highlight topics which you want to get more detailed information about posts". And then create a blog post with a list of all topics in the semantic web category like the other post.<br>+Uldis: One of the next things we can do is: take a topic or category hierachy from the original post and re-create them for this target post - or in fact we shouldn't only be thinking in terms of post, there are other kinds of objects, we could even just export SKOS categories and automatically create a category tree on another blog. Maybe we don't even need to import it - but just get the category hierachy from the source blog and display a user interface that says: "this is the category hierachy, highlight topics which you want to get more detailed information about posts". And then create a blog post with a list of all topics in the semantic web category like the other post.<br> -> We should just make richer and better use of all this metadata we are generating. -> We should just make richer and better use of all this metadata we are generating. -Antoher intersting direction: trying to import SIOC data into an existing CMS - which is probably a primary use case for data portability. <br>+Another interesting direction: trying to import SIOC data into an existing CMS - which is probably a primary use case for data portability. <br> Danbri: blog comments get lost. Use SIOC of create a life stream(?) archive. Danbri: blog comments get lost. Use SIOC of create a life stream(?) archive. -Sioc is dangerously close to the RSS and Atom space that it is very hard to justify outside of our community exactly how it differs. long term archive is probably the story why it is worthwhile.+SIOC is dangerously close to the RSS and Atom space that it is very hard to justify outside of our community exactly how it differs. long term archive is probably the story why it is worthwhile. -<br>Richard prefers this phrasing for the use of SIOC to copy content as "long term archiving of everything you've ever produced on the web". So you have your data in a secure place in case antoher service goes down +<br>Richard prefers this phrasing for the use of SIOC to copy content as "long term archiving of everything you've ever produced on the web". So you have your data in a secure place in case another service goes down. To some extent that is the mashup/aggregation scenario again: how to integrate discussions from all over the web and provide global overview. -> Find value in the aggregate. To some extent that is the mashup/aggregation scenario again: how to integrate discussions from all over the web and provide global overview. -> Find value in the aggregate. Line 65: Line 65: How important is it to achieve Data Portability? How important is it to achieve Data Portability? -Danbri: Users of social networking sites are on average registered on 1,6 of these sites. So the vast majority of people use only one site. From that point of view it is not a great big deal to type your profile in again. +Danbri: Users of social networking sites are registered on 1,6 of these sites on average. So the vast majority of people use only one site. From that point of view it is not a great big deal to type your profile in again. But many social features are appearing also on last.fm, flickr, dopplr and so on - But many social features are appearing also on last.fm, flickr, dopplr and so on - if you include those and ask users "which of these do you use", you get a lot more usage - there are far more internet users with profiles on many of these sites. if you include those and ask users "which of these do you use", you get a lot more usage - there are far more internet users with profiles on many of these sites. Line 72: Line 72: Danbri expects to see relatively soon the ability to connect friends between differnt sites: "through xmpp or open social, whatever the mechanism, it will get manifested in foaf or xfn eventually. similar for sioc." Danbri expects to see relatively soon the ability to connect friends between differnt sites: "through xmpp or open social, whatever the mechanism, it will get manifested in foaf or xfn eventually. similar for sioc." -Uldis: While there is maybe not such a generic way how you move this sort of data or at least notification of facts that data has appeared between different sites, many sites do implement their own versions of activity streams, they pull in a user's activity form other sites. Like jaiku would pull in from twitter and flickr and whatnot.+Uldis: While there is maybe not such a generic way how you move this sort of data or at least notifications of the fact that data has appeared between different sites, many sites do implement their own versions of activity streams, they pull in a user's activity form other sites. Like jaiku would pull in from twitter and flickr and others. -Richard: this is a very interesting thing as it relates to the topic of friend feeds. You publish stuff in many different places, so there is the problem: How can I see all of this in a single place? How can I have one application use the notifications of the stuff happening on another application? Zhat is something SIOC is very suited for.+Richard: this is a very interesting thing as it relates to the topic of friend feeds. You publish stuff in many different places, so there is the problem: How can I see all of this in a single place? How can I have one application use the notifications of the stuff happening on another application? That is something SIOC is very suited for. Danbri suggests maybe using IFrames to display a bit of information from foaf and xfn when you come across a page of someone that you are friends with - that is a very widgety model. Danbri suggests maybe using IFrames to display a bit of information from foaf and xfn when you come across a page of someone that you are friends with - that is a very widgety model. Line 80: Line 80: == BSCW, synchronization and authentication == == BSCW, synchronization and authentication == -Uldis: Talking about activity streams and notifications. In essence we could formulate it as that you are aggregating or collecting meta-data, as opposed to collecting actual content, we are mainly collecting meta data or information about the fact that something has been created. +Uldis: Talking about activity streams and notifications: In essence we could formulate it as that you are aggregating or collecting meta-data, as opposed to collecting actual content, we are mainly collecting meta data or information about the fact that something has been created. Deirdre explains that also in BSCW they are not actually replicating the content. Deirdre explains that also in BSCW they are not actually replicating the content. Line 89: Line 89: Deirdre: it is a lot easier as opposed to copying the whole content, which not really necessary, probably the user is not going to look at every single item in the folder. So why bother copying all that and going through all that overhead. Deirdre: it is a lot easier as opposed to copying the whole content, which not really necessary, probably the user is not going to look at every single item in the folder. So why bother copying all that and going through all that overhead. -Uldis: In a enterprise environement you are quite confident that the servers will be there and have good lines between them, so that you can reliable deliver the content.+Uldis: In a enterprise environment you are quite confident that the servers will be there and have good lines between them, so that you can reliable deliver the content. Deirdre: also for the moment the data is quite small and everything is in a closed environment. Deirdre: also for the moment the data is quite small and everything is in a closed environment. Line 100: Line 100: Is there a way to somehow make a standard atuhentification process? Because with all this data exported and then importers and viewers being there, there has to be some kind of authentification middle-man, but at the moment this is done proprietary. If you want to access data from one cwe then you have to have the auth for that and another for the next cwe. that's an unresolved issue. Is there a way to somehow make a standard atuhentification process? Because with all this data exported and then importers and viewers being there, there has to be some kind of authentification middle-man, but at the moment this is done proprietary. If you want to access data from one cwe then you have to have the auth for that and another for the next cwe. that's an unresolved issue. -danbri: Now everything is all openId and openAuth, we should look at that. We are also looking at OAuth, which is a very small leightweight piece that got traction in the big web site world. The analogy is apparently a valet key - it is a restricted key like: they park your car outside the hotel and you give a key that makes your car explode if it gets stolen - or it only goes at a certain speed and only goes so far. It is a restricted token.+danbri: Now everything is all openId and openAuth, we should look at that. We are also looking at '''OAuth''', which is a very small leightweight piece that got traction in the big web site world. The analogy is apparently a valet key - it is a restricted key like: they park your car outside the hotel and you give a key that makes your car explode if it gets stolen - or it only goes at a certain speed and only goes so far. It is a restricted token. -danbri:We thought about a mechanism - if we have a sparql store and you re-write the incoming queries to put in more constraints to make sure you can only look at a part of the graph that you are allowed to that corresponds to your OAuth credentials. We think we can implement that mechanism - it's not going to be efficient, but it can be wrapped around on a sparql store. +danbri: We thought about a mechanism - if we have a sparql store and you re-write the incoming queries to put in more constraints to make sure you can only look at a part of the graph that you are allowed to that corresponds to your OAuth credentials. We think we can implement that mechanism - it's not going to be efficient, but it can be wrapped around on a sparql store. danbri: One basic scenario for OAuth: giving a printing service access to your private pictures on flickr. danbri: One basic scenario for OAuth: giving a printing service access to your private pictures on flickr. Line 113: Line 113: * keeping your traffic to more or less what was said in the channel, may lose some of the actions. * keeping your traffic to more or less what was said in the channel, may lose some of the actions. * channel discovery, also doing a bit of social network analysis over channel membership. who talks, who is a member, who works in this channel. E.g.: How tightly connected is the debian world to the linux group or the sem web group. Do some pathfinding between those, finding the connected people. * channel discovery, also doing a bit of social network analysis over channel membership. who talks, who is a member, who works in this channel. E.g.: How tightly connected is the debian world to the linux group or the sem web group. Do some pathfinding between those, finding the connected people. -This is quite important for the jabber scene. It is realatively easy to find irc channles, not really easy in multi user chat and jabber xmpp world. if we had this kind of markup we could have a groups called jabber chat room and their homepage. saying_ there is a chat channel and its homepage is there. we could mix that whith foaf and skos and all kinds of things. +This is quite important for the jabber scene. It is relatively easy to find irc channles, not really easy in multi user chat and jabber xmpp world. if we had this kind of markup we could have a groups called jabber chat room and their homepage. saying: there is a chat channel and its homepage is there. we could mix that with foaf and skos and all kinds of things. -Danbri: We are logging swig and foaf channel and we get an rdf file and an html file sperate - doing it in RDFa would give it to us in one document - it is quite tempting to do that.+Danbri: We are logging swig and foaf channel and we get an rdf file and an html file separate - doing it in RDFa would give it to us in one document - it is quite tempting to do that. Danbri: SIOC is the natural home for all of that. Danbri: SIOC is the natural home for all of that.

Replacing page with 'Image:sdow.png == Social Data on the Web workshop (SDoW2008) == See also: http://sdow2008.semanticweb.org/ This wiki page is only for organizers.'

←Older revision Revision as of 13:53, 5 May 2008 Line 1: Line 1: [[Image:sdow.png]] [[Image:sdow.png]] -== Aim and scope of the workshop ==+== Social Data on the Web workshop (SDoW2008) == -The 1st Social Data on the Web workshop (SDoW2008) co-located with the 7th International Semantic Web Conference ([http://iswc2008.semanticweb.org/ ISWC 2008]) aims to bring together reseachers, developers and practitioners involved in semantically-enhancing social media websites, as well as academics researching more formal aspect of these interactions between the Semantic Web and Social Media.+See also: http://sdow2008.semanticweb.org/ -Since its first steps in 2001, many research issues have been tackled by the Semantic Web community such as data formalism for knowledge representation, data querying and scalability, or reasoning and inferencing. More recently, Web 2.0 offered new perspectives regarding information sharing, annotation, and social networking on the Web. It opens new research areas for the Semantic Web which has an important role to play to lead to the emergence of a Social Semantic Web that should provide novel services to end-users, combining the best of both Semantic Web and Web 2.0 worlds. To achieve this goal, various tasks and features are needed from data modeling and lightweight ontologies, to knowledge and social networks portability as well as ways to interlink data between Social Media websites, leveraging proprietary data silos to a Giant Global Graph. +This wiki page is only for organizers. - + -This workshop is intended to bring together people and organisations interested in applying Semantic Web technologies to social media sites.+ - + -== Workshop Organization ==+ - + -The workshop will be co-located with the 7th ISWC, and will be help on the 26th or 27th October 2008.+ - + -The workshop will consist of:+ -* Opening session: This will permit introduction of the workshop topics, goals, participants, and expected outcomes.+ -* Keynote speakers: We expect to attract at least two keynote speakers for the workshop, both from academic and industrial fields.+ -* Brief presentations: Contributors will be asked to submit a short position paper on how their contribution is relevant to semantically-enhanced social media sites, accepting questions during their presentations to support a more interactive environment.+ -* Parallel demonstration and poster session: This will involve all participants making their contribution available for examination. Posters will be accepted for contributors who are at an earlier stage of research.+ -* Discussion session: A panel-led discussion entitled "Enhancing social media sites with Semantic Web technologies: open problems and future research directions" will close the workshop.+ -* Communication: Networked communication will be encouraged during the Workshop using an IRC backchannel and other services enriched with Semantic Web capabilities.+ - + -== Call for papers ==+ - + -We encourage the submission of position papers, demonstrations and posters which will describe research proposals or implementations that deal with (but are not limited to) the following topics of interest:+ - + -* Accessing Social Data+ -* Data Portability+ -* Emerging semantic platforms for the Social Web+ -* Linked Data on the Social Web+ -* Ontologies for the Social Web+ -* Policies, authentication, security, and trust within collaborative scenarios+ -* Producing Semantic Web data from existing social software applications+ -* Reasoning on the Social Web+ -* Semantic markup solutions+ -* Social Knowledge Management+ -* Combine Web 2.0 and Semantic Web strategies and technologies+ -* Social and semantic bookmarking, tagging and annotation+ -* Social networks and social wikis + - + -The following types of contributions are welcomed:+ -* Position papers+ -* Posters+ -* Demos+ - + -Paper submissions will have to be formatted in the style of the Springer Publications format for Lecture Notes in Computer Science ([http://www.springer.com/computer/lncs?SGWID=0-164-2-72376-0 LNCS]). + -Submissions will have to be made using EasyChair Conference System, and proceedings of the papers will be provided through the CEUR online service.+ - + -== Important dates ==+ - + -* Paper, demos and posters submissions: 25 Jul+ -* Notification: 5 Sep+ -* Camera ready papers: 26 Sep+ -* Camera ready proceedings: 03 Oct+ - + -== Organizing Committee ==+ - + -* John Breslin, DERI, NUI Galway, Ireland - http://johnbreslin.com+ -* Uldis Bojārs, DERI, NUI Galway, Ireland - http://captsolo.net+ -* Alexandre Passant, LaLIC, Université Paris-Sorbonne, France - http://apassant.net+ -* Sergio Fernández, Fundación CTIC, Spain - http://wikier.org+ - + -== Program Committee ==+ - + -* TBA+

New page: Recovering Deleted/Lost/Missing Data From Novell Servers This paper discusses data loss from Novell server disks along with methods to recover and restore the lost data. Effective reco...

New page

Recovering Deleted/Lost/Missing Data From Novell Servers
This paper discusses data loss from Novell server disks along with methods to recover and restore the lost data.
Effective recovery of lost data requires an understanding of data management techniques used by NetWare …quot; this includes understanding NetWare file systems and NetWare volume types. Basic awareness about various data loss scenarios along with NetWare's built-in defense mechanisms against data loss is also required.
In instances of data loss, it is recommended that the affected server disk not be used further and be immediately disconnected…quot; the correct way to recover data from failed disks is to connect these disks to healthy working disks and initiate the recovery process from the healthy disks. Thorough knowledge on the connection of such disks is a must during a recovery process.
Finally, robust data recovery software is needed to ensure complete and effective recovery.
This document is a discussion on all of the above. A generic description about the working of data recovery software is also outlined, followed by a short case study (using Stellar Phoenix Novell) to help users better understand the recovery process. This document assumes that the reader has little or no knowledge of data recovery and only very basic knowledge of Operating Systems, hard disks and NetWare.
Table of Contents
Data Management in Novell NetWare
1.1 NetWare server disks and file systems
1.2 NetWare Volume types
1.3 How data gets lost
1.4 Data recovery methods provided by NetWare & its limitations
Setting up the hardware (for a recovery process)
2.1 Basic Do's and Don'ts
2.2 Connecting IDE disks
2.3 Connecting SCSI disks
2.4 Finishing up
Using data recovery software (Stellar Phoenix Novell)
3.1 How data recovery software work
3.2 Case Study …quot; Using Stellar Phoenix Novell to recover data
3.2.1. How to recover deleted files
3.2.2. How to recover missing/lost NetWare partitions
3.2.3. How to recover compressed volumes
Conclusion

1. Data Management in Novell NetWare
1.1 NetWare server disks and File Systems
A NetWare server disk consists of 2 basic partitions. The first is a small MS-DOS partition used to boot the computer and then load the NetWare OS. The second partition contains the NetWare OS files.

Fig: Structure of NetWare server disk
The NetWare partition (shown in blue in the figure above) consists of 2 basic divisions: a Hot Fix Redirection Area and a Data Area.
The Hot Fix Redirection Area contains sectors which act as substitutes for bad sectors on the hard disk. Before writing data to the disk, NetWare verifies whether the target sectors are in good condition …quot; if they are not, the data is redirected to the sectors in the Hot Fix Area. The Hot Fix Area typically comprises 0.2 - 2% (user-specified) of the disk space. This area consists of two parts: one being a table which maps bad sectors in the data area to corresponding sectors in the Hot Fix Area, and the other comprising the actual sectors which act as substitutes for bad sectors.

Fig: Structure of a NetWare Partition and its Hot Fix Redirection Area
The Data Area consists of a Volume Table (which contains detailed information of the volumes such as name, size, block size, segmentation, and other details) and one or more logical Volumes. The first volume in any NetWare partition is the SYS Volume. Shown below is the Volume distribution in a typical server disk:

Fig: Distribution of Volumes on a server disk
The methods and data structures used to organize files on a Volume are known as its File System. NetWare file systems have evolved over time with NetWare 2.x using NWFS 286, NetWare 3.x and 4.x using NWFS 386, while NetWare 5.x and later uses the NSS (Novell Storage Services) file system.
NWFS
NWFS can be thought of as a modified FAT (MS-DOS/Windows) file system. It used to come in both 16-bit (NWFS 286) and 32-bit (NWFS 386) versions, and has now been superseded by the 64-bit NSS (Novell Storage Services) file system.
NetWare 4.x and earlier had volumes formatted to the NWFS file system which is similar to Microsoft's FAT 32 file system. Distributed across a NetWare Volume are certain blocks which contain 2 important data structures that manage files and directories …quot; these are DET (Directory Entry Table) and FAT (File Allocation Table). This is similar to the Directory Table and File Allocation Table used in Microsoft's FAT file system.
The DET contains a list of all files and directories in the volume including details such as name, size, attributes, and so on. The FAT is a table that contains information about where on the Volume various files are stored, which blocks they occupy, and so on …quot; all files can be located using the FAT. 2 copies each of the DET and FAT are maintained (for the sake of redundancy). The DET and FAT blocks are distributed throughout the Volume.

Fig: Structure of a NetWare Volume
In NWFS, the Directory entries are recorded in a Table, and the File Allocation entries are maintained using Linked Lists.
NSS
Unlike NWFS (which manages data using the DET and FAT), NSS is a Journaling file system. This means that any change made to a file, instead of being directly updated to the DET and FAT as in NWFS, is first logged in a journal before the actual change is made. Only the files metadata (data about the file such as name, size, location, and so on) is logged and not the actual file data. Therefore, if the disk crashes while some write processes are running, the OS knows which processes to redo on rebooting.
All structured data on a Volume formatted to NSS is organized using the B-Tree (Balanced Tree) algorithm. The B-Tree is a tree data structure and is very efficient for use in file systems and databases. Two of the B-Trees found in NSS are the Name Tree B-Tree which manages a Volume's directory structure (analogous to the DET in NWFS) and the Beast B-Tree which manages all file's metadata (analogous to the FAT in NWFS).
A B-Tree is very efficient with insertions and deletions which is why it is used in file systems and data bases
NSS v/s NWFS with respect to Data Safety and Protection
NSS is far superior to NWFS when it comes to data safety and protection
NSS NWFS
Error Correction Presence of journal simplifies error correction/recovery Entire volume must be scanned (VREPAIR) to determine errors
File Snapshot Backup of open files No such support available
Pool Snapshot Backup of block in current state No such support available
RAID Support Supports RAID 0,1,5,10,15 Supports RAID 0,1,10
Data Shredding
(Data Erasure) Permanent data removal
(overwriting over 7 passes) No such support available
1.2 NetWare Volume Types
Data storage in Novell servers can be optimized using configurations such as Compressed Volumes, Sub-allocated Volumes, and Segmented Volumes. Data recovery software usually recovers data in its raw form, as they appear on the disk, disregarding any of the above configurations they may be in. Thus, the recovery of such data back to its original form presents a problem. Before we look at how such configurations can be retained even after recovery, let us first understand how data is arranged in such volumes.
Compressed Volumes
Data Compression refers to the process of storing data in a format that requires less space than usual. Compressing data is the same as packing data …quot; reducing the amount of electronic 'space' data takes up, and compressed data usually consumes 2 to 4 times fewer bits. Typically, it is infrequently used data which is automatically compressed to save space.
Methods of compressing data include replacing multiple blank spaces with a character count, or replacing redundant data with shorter stand-in 'codes'.
Volumes can be compressed in NSS using the console command:
nss /Compression=(Volume Name) OR nss /Compression=all
Volume compression statistics can be viewed using the command:
CompScreen
Note: Volume compression is an irreversible process.
A listing of all Novell compression capabilities is available at: http://www.novell.com/documentation/nw65/index.html page=/documentation/nw65/nss_enu/data/bpwtq9o.html
Sub-allocated Volumes
Sub-allocated volumes are a new feature and have been implemented in NetWare 4.x to overcome the problem of wasted disk space. These volumes are different from regular Novell volumes because they are more of a 'virtual' volume …quot; one that is not 'visible', but exists nevertheless.
Sub-allocation refers to multiple file-endings sharing disk blocks. The data in a file is stored in the hard disk in multiple blocks, each block being a collection of sectors of standard size. There are, almost always, some unutilized sectors at the end of a file. These 'under-allocated' sectors of different files are combined together to form a separate 'virtual' volume.
For Example: Say, a volume stores data in blocks of 4 sectors, each sector having size = 512 bytes. Now, if there is a file containing 520 bytes of data, the first sector is entirely occupied and the remaining 8 bytes go into the second sector. This leaves 2 entire sectors unused. These unutilized sectors of different files are combined together to form a separate sub-allocated volume, thus saving space and optimizing data storage.

Fig: Space optimization using Sub-allocated volumes
The sub-allocated volume is virtual - it does not show up as a separate logical volume. It is internally used by the Operating System (NetWare 4.x) to optimize disk space utilization.
Sub-allocation was a useful feature in NWFS which uses 64K blocks. NSS, which uses 4K blocks, do not have much use for sub-allocation.
Segmented Volumes
Segmented volumes refer to volumes which have had their size increased by taking disk space from another area. This space is usually taken from another disk, resulting in the Volume existing in 'segments' across different disks, and hence the name segmented volumes. Segmented volumes add flexibility and are particularly useful in cases of important volumes needing additional disk space. A volume may be spanned across as many as 32 hard disks.

Fig: Volume X is a segmented volume spanned across 3 hard disks
For Example: Say, a disk consists of 2 volumes, one of which is almost full up and needs additional space. NetWare uses a feature using which disk space, usually from a separate disk, can be added to an existing volume.
While Segmented Volumes do make for increased volume size, the downside is that the volume gets fragmented.
1.3 How data gets lost
Data loss can be just a click away and usually occurs when least expected. Some of the leading causes of data loss:

(Source: www.protect-data.com/information/statistics.html)
The biggest factor leading to data loss is hardware malfunction. Hard disks are mechanical devices and therefore fall victim to wear and tear …quot; estimates put the average life of a hard disk at 3 years and it's easy to see why - picture a series of disk platters rotating at 10,000 times per minute (over a 150 times a second!) with disk heads moving over them, separated at submicron distances. The margin for error, for mechanical movements inside the disk, is minimal. Even slight vibrations can unsettle alignments, and dropping a disk to the floor can be thought of as a minor catastrophe …quot; in fact, Head/Media collisions account for a large percentage of hardware malfunctions. Add to this other factors such as settling of dust particles on magnetic surfaces, sudden temperature variations, electronic (circuit board) failures, Controller failures, power surges, etc and one can understand the relatively short life span of hard disks …quot; a hardware malfunction can occur at any time and is sometimes beyond the control of the user.
Human error (accidental deletion/format, incorrect usage of software, etc) and software malfunction (corruption by virulent software, configuration complexity, improper backups, etc) are other major factors leading to data loss. While virus attacks are an issue with other Operating Systems (most notably Windows), this is not much of an issue in NetWare.
1.4 Data safety methods provided by NetWare & its limitations
Although NWFS has limited data safety/protection methods, NSS comes with useful features some of which are:
Journaling -> This significantly reduces the number of errors that happen when an operation is interrupted unexpectedly (resulting from occurrences such as a power failure or system failure).

File Snapshot -> This feature allows transient copies of open files to be made. Taking advantage of this, backups of even open files are possible. This feature can be activated using the console command:
nss /FileCopyOnWrite=(Volume Name) OR nss /FileCopyOnWrite=all
For a complete description of this feature, visit:
http://www.novell.com/documentation/nw65/index.html page=/documentation/nw65/nss_enu/data/ajhv67e.html

Pool Snapshot -> This feature allows you to take a snapshot of a data pool as it exists, including open files. This can be helpful for backup, maintenance, and restoration of lost data.
A snapshot can be taken using the console command:
mm snap create (snappool)(datapool)(snapname)
A previously taken snapshot can be activated using the console command:
mm snap activate (snapname)
A more extensive list of Pool Snapshot capabilities is given at:
http://www.novell.com/documentation/nw65/index.html page=/documentation/nw65/nss_enu/data/br18vpz.html
Even though these features offer data safety up to a certain level, the one big limitation for these features above is that all backups are metadata-based only. This means that a data disaster on a big scale, with a files actual core data getting lost/deleted, cannot be resolved using these features. What is needed in such a scenario is a complete scan of the disk surface to look for deleted/missing entries. This can only be done by advanced data recovery software and is a must-use when the above methods fail.
2. Setting up the hardware (for a recovery process)
2.1 Basic Do's and Don'ts
When data is seemingly 'deleted' from a disk, the actual data is not removed. Only the reference information of the file is modified so that the space that a file occupies on the disk is marked as 'free-space' and is made available for writing on. But until that space is actually overwritten by other data, the original data remains intact and is therefore recoverable.
Since 'lost' data being overwritten is a major concern, it is imperative that the volume containing lost data is not used at all. Even leaving the server on, without any activity, causes many tiny files to be written/updated every few seconds. This increases the chance of lost data being overwritten and reduces the possibility of total data recovery. Therefore, after any instance of data loss …quot; be it a single deleted file or a total disk failure …quot; it is preferable to disconnect the affected server disk.
Following episodes of data loss:
Do's
Turn off the machine immediately
Disconnect the server disk
Don'ts
Never restart the machine in the hope that this will solve the problem …quot; a restart involves many files being generated which may result in overwriting of lost data which will in turn reduce the possibility of total data recovery
If you hear clicking/grinding noises coming from the storage media, then it is a hardware failure …quot; do not continue using the disk. It is preferable to disconnect the server disk immediately. In such cases, even data recovery software may not fully be able to recover lost data …quot; it needs to be turned over to data recovery specialists
If there is intense physical damage to the disk (such as a power surge resulting in smoke or sparks), do not experiment with the media …quot; this needs examination by experts
Do not, under any circumstances, open up the disk since exposing the platters may damage the disk beyond repair. The disk is to be opened up only as a last resort and that too by professionals working in a clean-room
If the data recovery software allows installation on NetWare, do not install the software on the Volume which is affected. While it is possible to install the software on a Volume other than the one which is affected, it is always preferable to connect the affected disk to a working disk and then install the recovery software to the working disk
The ideal method to recover lost data is to connect the affected disk to a healthy disk. Data recovery software is then installed on to the healthy disk after which the software detects and does a scan of the affected disk.
The following section describes the connection of the disks.
2.2 Connecting IDE disks
It is best not to involve the server disk for recovery software installation during a system recovery. It is recommended to install the software on a Windows OS 'host' disk, and then connect the affected server disk to it (to understand why, read section 3 labeled 'Using Data Recovery Software').
This section discusses the connection of IDE (also known as ATA) disks.
Disks with IDE interfaces can be connected as:
'Slave' on the Primary channel OR
Either 'Master' or 'Slave' on a secondary channel
This implies that the disk with lost data can be connected in any way to a working disk, as long as the system does not boot from the affected disk.
Configuring a disk to be either 'Master' or 'Slave' involves setting the jumper pins at the back of the hard disks.
This is what the back of a disk looks like:
IDE Cable Connector
One end of this connector goes into the back of the disk, and the other into the motherboard.
Jumper Pins
Out of the given sets of jumper pins, one of the sets needs to be 'jumpered' thus giving the disk its configuration. The diagram for the jumper pin configuration should be given in a sticker on the disk. If the failed disk is being connected on the primary channel, it needs to be connected in the 'slave' configuration. Use tweezers to change the switch position. There is a notch that prevents incorrect insertion.
Power Cable Connector
The power cable goes in here.


This is what an IDE cable looks like:
Connecting disks to the cable
An IDE cable has one end connected to the motherboard, and the other end usually goes into the disk configured as 'Master'. The working disk which has recovery software installed in it should be the 'Master'. The plug in the middle usually has the disk configured as 'Slave'. If connected in the primary channel, the damaged disk needs to be 'jumpered' to be a 'Slave' and then connected to this middle plug.

After the jumper pins have been set and IDE cable connected at the correct points, the disks need to be installed into its case …quot; simply screw in the disk to secure it to its case. The next step is to make sure that the computer accepts the disks.
2.3 Connecting SCSI Disks
While disks with IDE interfaces have gained popularity with home PC users because of their low cost, SCSI interface disks are common with servers and high performance workstations and/or peripherals. The most visible difference between IDE and SCSI disks is that a single SCSI adapter can handle as many as 7 or 15 devices at a time …quot; because of this, setting the Jumper Pins at the back of a SCSI disk are a little more complicated than behind an IDE disk.
In connecting SCSI disks appropriate jumper pins have to be set to assign SCSI ID's, configure termination, etc. For data recovery purposes the only thing to remember while connecting the disks is to connect the SCSI disk with lost data as:
'Slave' on the Primary channel OR
Either 'Master' or 'Slave' on a secondary channel
This implies that the disk with lost data can be connected in any way to a working disk, as long as the system does not boot from the affected disk.
This is similar to disks with IDE interfaces (see above …quot; Connecting IDE disks - for connection details). The jumper pin setting information should be detailed in the sticker on the disk itself. After connecting the SCSI disk in the appropriate configuration, the next step is to make sure that the computer accepts the disks.
2.4 Finishing Up
The BIOS usually automatically detects new hardware - this is so because the disk detection feature is set to 'AUTO' by default. However, it is still better to enter the BIOS and confirm whether the appropriate disk channel(s) are set to 'AUTO'.
There are different methods of entering the BIOS for different motherboards. It is usually done by pressing the 'F2' key or the 'Delete' key just after starting the computer (To find out the exact keystroke(s) to enter the BIOS, visit: http://www.numonics.com/support/bios.htm).
In the BIOS, navigate through the main menu and go the Hard Disk AutoDetect feature


In the Hard Disk setup window, make sure the channel that the disk is connected to, is set to 'AUTO'. For Example: If the disk with lost data is a 'slave' on the primary channel, make sure that this option is set to 'AUTO'.

After making the necessary changes, remember to save changes before exiting BIOS.
3. Using Data Recovery Software (Stellar Phoenix Novell)
3.1 How data recovery software work
When a file is seemingly 'deleted' from a disk, the actual data is not removed. The only thing that is done is modification of the file's entry in the directory table/tree such that the space on the disk that the file occupied is now labeled as free space. Thus, the file still exists on the disk - but the file system is blind to it. This is much like a house without an address - it exists, but there is simply no way to go to it. Permanent deletion occurs only after the file's actual data is overwritten by other data.
Data recovery software, such as Stellar Phoenix Novell, take advantage of this fact and use it to recover lost data.
When a file is 'deleted', only the first few bytes of that files entry in the Directory Table/Tree are modified to make the file system recognize that file as deleted. Data Recovery software scan the Directory Table/Tree looking for modified entries which have been labeled as 'deleted'. When it finds one, it verifies whether the clusters occupied by this file have been left untouched or whether the clusters have already been overwritten by a new file. If the clusters are still 'free' and have not been reallocated, then this file is recoverable.
This is the reason why an affected disk should be disconnected and not left running as tiny files keep getting generated/updated every few seconds, thus increasing the likelihood of crucial data getting overwritten …quot; it is easy to understand then, why even something as simple as a system restart could turn out to be catastrophic.
Sometimes, however, even the Directory Table/Tree may be corrupt or may have been overwritten. In such cases, data recovery software go into advanced mode and scan the disk cluster by cluster. When the software finds a file, it verifies whether that file's entry exists in the Directory Table/Tree …quot; if it does, then the file is obviously 'visible' to users. However, if an existing file on the disk does not have an associated entry in the Directory Table/Tree, then it is considered to be a lost/missing entry and the software will display this as a 'found' file.
Thus, data can be recovered either by analyzing the Directory table/tree of a file system (to look for deleted files), or by doing a cluster by cluster search of the physical disk (to look for lost/missing files). These two options are given different names (like 'Standard Search' and 'Advanced Search') by different data recovery software.
The next section discusses the recovery process of deleted files and missing/lost NetWare partitions using Stellar Phoenix Novell.
Problems faced in recovering data from compressed/sub-allocated/segmented volumes:
Data recovered from volumes which are compressed, sub-allocated, or segmented presents a few problems.
Compressed Volumes
Data recovery software recover lost data exactly as it was stored in the disk, and the recovered data may be compressed. Data must be decompressed before it can be used. This gives rise to the problem of recovered data being unusable (because it is in a compressed format).
Sub-allocated Volumes
Sub-allocated Volumes are virtual and not 'visible'. Since data recovery software do a physical scan on the hard disk, recovery of lost data from sub-allocated sections of a volume is a problem.
Segmented Volumes
Regular data recovery software can usually scan only one disk at a time, but segmented volumes may be spanned across multiple disks, which is what creates the difficulty in the recovery of segmented volumes.
Effective data recovery software for NetWare requires advanced modules to fully recover data from such volumes while at the same time maintaining the original configuration …quot; ensure that the data recovery software you purchase can recover data from all these volume types
3.2 Case Study …quot; Using Stellar Phoenix Novell to recover data
All data recovery software have a trial version available which lets users get a good approximation of how effective the software can be. The trial version scans the disk using both Standard and Advanced modes and displays a list of all recoverable data. However, the full version will be needed to actually restore/save that data.
If you have lost data to be recovered, try downloading Stellar Phoenix Novell from www.stellarinfo.com/netware-data-recovery.htm and then install the software to a healthy working hard disk with the Windows Operating System in it. Stellar Phoenix Novell is a win32 executable and needs to be installed to a Windows based disk.
Run setup.exe from among the downloaded files …quot; the software is installed, by default, to a Stellar folder in Program Files. Additionally, usual entries to the Programs group (Start -> Programs) and the Add/Remove Programs list (in Control Panel) are also made.
Note: Make sure you are logged in as administrator before initializing setup - the software installation requires local administrative rights in Windows NT/2000/XP.
The affected server disk should be connected to this 'Windows' disk as a secondary (for details, read Section 2: Setting up the hardware).
Running the software will display the user interface containing a Toolbar which contains all necessary buttons to operate the software, from initiating the recovery process to saving the lost data.
Limitation of Stellar Phoenix Novell: NetWare 6.5 and later has an optional Encrypted Volume Support (EVS) feature using which it is possible to encrypt data stored on any newly created volume (the SYS volume is the only volume that cannot have the EVS feature in it). This is done mostly for protection of data against theft. Although data in EVS volumes are in an encrypted format, they work seamlessly with all applications. To learn how to create an encrypted volume, visit: http://www.novell.com/documentation/nw65/index.html page=/documentation/nw65/nss_enu/data/bq2y6nb.htm.
Stellar Phoenix Novell cannot recover data from such encrypted volumes.
For further information on the workings of this software, read the product's Knowledge Base articles at: http://stellarinfo.com/esupport/users/kb.php category_id=13
3.2.1. How to recover deleted files
NWFS
Click the Select Physical Disk button ( ) on the Toolbar to begin the recovery process. The software displays a list of all the hard disks it can detect, including the affected server disk. Select the server disk and then click OK to continue.

Select the NetWare partition


Select the Volume


The software looks for deleted entries in the selected volume and displays the result in a simple tree structure.


If the standard search fails to turn up the desired results, the software can be made to search in an advanced mode (read Section 3.1: How data recovery software work) by changing the Linking Mode to Contiguous (Tools->Change Linking Mode->Contiguous Mode).

Select any number of files to be recovered and then click the Recover Selected button to restore the data. Alternatively, clicking the Recover All button will recover all displayed files. (The Recover Selected/All feature is only activated in the full version …quot; it is not available in the trial version).
NSS
The steps are similar to the ones shown above. They are:
Select the physical disk

Select the NSS Pool

Select the Volume

The software looks for deleted entries in the selected volume and displays the result in a simple tree structure after which any number of files can be recovered.
3.2.2.How to recover missing/lost NetWare partitions
The process of recovering of missing/lost NetWare partitions is similar to the steps shown above in 3.1 How to recover deleted files. The one difference is that instead of specifying the Pool and/or Volume to recover, you need to choose the hard disk to recover from …quot; this is because the software will scan the entire disk to look for missing/lost NetWare partitions.
NWFS
Use the 'Search NetWare Partition' option and then select the physical disk which contains the lost/missing partition.

The software searches the selected disk for all partitions. Since the search is intensive, it may take some time to complete. After the search is over, the software will display a list of all found partitions

Select the partition which contains the missing/lost volume

Select the Volume from which data needs to be recovered

The software then analyses the volume's data structures and file system attributes and displays the result in a simple tree structure after which any number of files may be recovered
NSS
The method of recovering missing/lost NSS Volumes is similar to the steps given above. Moreover, Stellar Phoenix Novell also has an additional feature in which NSS Pools can be searched for, and recovered.
Use the 'Advanced Search for NSS Volumes' or 'Advanced Search for NSS Pools' option and select the physical disk which contains the lost/missing Volume or Pool

The software does an intensive search and lists all the pools/volumes it can find

Select the Pool/Volume from which data needs to be recovered

The software analyses the relevant data structures and file system attributes and displays the result in a simple tree structure after which any number of files may be recovered
The steps laid out in the 2 sections above work for most Volume types, such as Segmented Volumes, Sub-allocated Volumes, etc and for most data loss events, such as Formatted Volumes, Re-partitioned Volumes, etc. However, recovery of compressed volumes is a slightly more complicated issue and is discussed in the next section.
3.2.3. How to recover compressed Volumes
(For NetWare 4.x and above)
If a Volume is given compression capability, then some data in the Volume will most likely be compressed. When such data is recovered, it is still in a compressed format and thus unusable. Thus, effective data recovery software need to have a decompression module in them to ensure full usability of the recovered data …quot; ensure that the recovery software you use has such a capability.
The use of recovery software's decompression utility is illustrated by the following steps, as followed in Stellar Phoenix Novell:
Recover the compressed data using the steps followed in sections 3.1 or 3.2 …quot; you now have recovered compressed data on a hard disk different from the affected server disk

Connect the disk with the compressed data to a Novell client machine …quot; this machine now has the compressed data in it

Log on as an Administrator to the Novell client and upload the compressed data to a Novell server disk …quot; the server disk now has the compressed data.
Note: The compressed data should be uploaded to an NWFS Volume on the server


Shut the server down, disconnect the server disk and connect it as a secondary to the disk with Stellar Phoenix Novell installed in it

Run the software, select the volume with the compressed data and then run the software's decompression utility on the compressed data (Tools -> Mark Compressed Files). The server disk's compressed data is now decompressed

Reconnect the server disk back into the server machine and boot this machine up

The decompressed data can now be transferred from the server to any client machine
This process does require some patience but is safest and yields best results if done systematically.
Data recovery software, as with most other software, come with their own unique additional features to ease the recovery process. Stellar Phoenix Novell comes with utilities such as File Filter, File Mask, Save Scan, Event Log, Find, and so on to streamline and facilitate the recovery process.
4. Conclusion
The data recovery process for Novell server disks can be summarized briefly as:
Disconnect the server disk

Install the data recovery software to a healthy working disk and connect the affected server disk to this working disk

Run the software to detect the server disk and display all its partitions/volumes

Select the disk/volume from which to recover data from and begin the disk/volume analysis

After all recoverable files are displayed, select the file(s) to be recovered and then restore them
In conclusion, if the server disk is immediately disconnected after an episode of data loss then the chances of total data recovery is very high. Data is never truly lost unless it is overwritten, and it is users that control writing to the disk …quot; hence, salvaging data, even in the event of a total disk failure, is not difficult at all. All that needs to be followed is a systematic approach.
The best practice in protecting data is regularly taking backups. While most Operating Systems today do have built in backup tools, these are not truly extensive. It is always better to go in for genuine 3rd party software (there are many good applications available today), whose development is dedicated to the task of protecting data. Most backup software today are fully automated and keep taking regular backups on their own without any user intervention …quot; even if the disk completely crashes, the machine can be booted using the software's disc (which contains the necessary boot files) after which the software accesses the backup and restores the system. [http://www.fixdisk.net/ 数据恢复]
Other effective data protection practices are hardware configurations such as RAID, periodic usage of good system checker software, reduction of dust around the work area, monitoring hard disk's for hardware faults (see Section 1.3: How data gets lost), effective cooling systems, updated anti-virus software, etc.
However, even backups (be it in the form of backup software or hardware configurations such as RAID) are not 100% foolproof and mishaps can occur in any situation. The only way to ensure complete data protection is a thorough knowledge of data storage and recovery processes …quot; this will provide users with small but extremely vital bits of information such as immediate disconnection of affected disks, connection of failed disks to healthy ones, recovery of data to safe locations, etc. Thus, in a nutshell, while data disasters may occur at any time, 100% data recovery is definitely possible.
Related Material:
To understand recovery of data even after it has been overwritten, read Gutmann's paper at: www.cs.auckland.ac.nz/~pgut001/pubs/secure_del.html [http://www.raidsos.org/ 数据恢复]

New page

== Natural Language Interface to SIOC Data ==

Use http://www.monrai.com/products/cypher/cypher_manual.html to build a natural language query interface to SIOC data.

A simpler query interface could probably be built with just regexps and pre-defined queries.

Recorded by: CaptSolo - 03-May-2008

== Offline Blog Editor using SIOC ==

Use SKOS to download category list/hierarchy from the blog engine

Can use SIOC to download existing posts

HTTP POST a blog post in SIOC in order to publish it on the blog

What is needed:
* SIOC import functionality, with support of HTTP POST and some authentication
* SKOS export
* add properties for indicating status of the post (private, draft, protected, ...) in SIOC
* ...

Reference material:
* http://www.blogger.com/developers/api/1_docs/xmlrpc_newPost.html

Recorded by: CaptSolo - 03-May-2008

== Blog Archives as a Book ==

A tool to collect [all] blog posts and create a PDF book out of it. There can be other variations too.

Reference:
* interesting ways to view a blog archive = http://www.sonsofskadi.net/extended-live-archive/
Recorded by: CaptSolo - 03-May-2008

←Older revision Revision as of 17:02, 2 May 2008 (2 intermediate revisions not shown.)Line 16: Line 16: # URIs for SIOC concepts # URIs for SIOC concepts -1. The ususal URL where the vBulletin forums, threads, posts and user profile pages can be retrieved. sioc:link and sioc:links_to always point to HTML pages, e. g.+1. This is the ususal URL where the vBulletin forums, threads, posts and user profile pages can be retrieved, e. g. http://boards.ie/vbulletin/forumdisplay.php?f=1 http://boards.ie/vbulletin/forumdisplay.php?f=1 http://boards.ie/vbulletin/showpost.php?p=54636504 http://boards.ie/vbulletin/showpost.php?p=54636504 - http://boards.ie/vbulletin/showpost.php?p=54636504+ http://boards.ie/vbulletin/member.php?u=31714 - +   +When there are <sioc:link> and <sioc:links_to> properties in the data, they always point to HTML pages. 2. The URIs for RDF/XML are constructed by appending ''sioc.php?'' followed by the type of the SIOC concept (site, forum, thread, post and user), the vBulletin ID of the object and optionally a paging parameter. To get e. g. the RDF data for thread #190769 we would append the following to the base URL <nowiki>http://boards.ie/vbulletin/</nowiki>: 2. The URIs for RDF/XML are constructed by appending ''sioc.php?'' followed by the type of the SIOC concept (site, forum, thread, post and user), the vBulletin ID of the object and optionally a paging parameter. To get e. g. the RDF data for thread #190769 we would append the following to the base URL <nowiki>http://boards.ie/vbulletin/</nowiki>: Line 36: Line 37: http://boards.ie/vbulletin/sioc.php?sioc_type=user&amp;sioc_id=1 http://boards.ie/vbulletin/sioc.php?sioc_type=user&amp;sioc_id=1 http://boards.ie/vbulletin/foaf.php?u=1 http://boards.ie/vbulletin/foaf.php?u=1  +  +The <rdfs:seeAlso> property is used to point to URIs like these 3. The unique identifiers for the SIOC concepts themselves are the same as the HTML pages displaying them for sioc:Site sioc:Forum sioc:Thread and sioc:Post. 3. The unique identifiers for the SIOC concepts themselves are the same as the HTML pages displaying them for sioc:Site sioc:Forum sioc:Thread and sioc:Post. -sioc:User and foaf:Person are exceptions as they are not considered information resources. Their URIs are constructed by appending #user or #person to their equivalent RDF/XML URIs:+sioc:User and foaf:Person are exceptions as they are not considered information resources. Their URIs are constructed by appending #user to the URI of the HTML user profile page or #person to the RDF/XML URI of the FOAF file: http://boards.ie/vbulletin/foaf.php?u=3289#person http://boards.ie/vbulletin/foaf.php?u=3289#person Line 182: Line 185: </sioct:BoardPost> </sioct:BoardPost> -When it come to sioc:User and foaf:Person there is an important distinction to make:+When it comea to sioc:User and foaf:Person there is an important distinction to make: A foaf:Person holds an online account. This online account is the sioc:User. So in our sense a user is not a person, but only an account that belongs to a person (and this person can have several user accounts on differnt sites.) A foaf:Person holds an online account. This online account is the sioc:User. So in our sense a user is not a person, but only an account that belongs to a person (and this person can have several user accounts on differnt sites.) A person uses her sioc:User account to create posts, so the triples denoting authorship are of the form sioc:User sioc:creator_of sioc:Post A person uses her sioc:User account to create posts, so the triples denoting authorship are of the form sioc:User sioc:creator_of sioc:Post

←Older revision Revision as of 16:40, 2 May 2008 Line 249: Line 249: * Submit to PEAR (?) * Submit to PEAR (?) * Allow external FOAF profiles * Allow external FOAF profiles  + * Allow API to output N3/turtle * What about sioc:topic VS sioc:subject (http://groups.google.com/group/sioc-dev/browse_thread/thread/6d8b7c7fdcd95acc) * What about sioc:topic VS sioc:subject (http://groups.google.com/group/sioc-dev/browse_thread/thread/6d8b7c7fdcd95acc) * Do not display properties if their value is not set {{Review}} * Do not display properties if their value is not set {{Review}}

←Older revision Revision as of 16:39, 2 May 2008 Line 10: Line 10: === OpenLink Data Spaces (ODS) Endpoints === === OpenLink Data Spaces (ODS) Endpoints === {{:OpenLink_Data_Spaces/Sparql_Endpoint}} {{:OpenLink_Data_Spaces/Sparql_Endpoint}}  + [[OpenLink_Data_Spaces|Read more... ]] [[OpenLink_Data_Spaces|Read more... ]] Line 23: Line 24: === WordPress === === WordPress === {{:Wordpress_Exporter}} {{:Wordpress_Exporter}}  + [[Wordpress_Exporter|Read more... ]] [[Wordpress_Exporter|Read more... ]] === DotClear === === DotClear === {{:DotClear_Exporter}} {{:DotClear_Exporter}}  + [[DotClear_Exporter|Read more... ]] [[DotClear_Exporter|Read more... ]] === Drupal === === Drupal === {{:Drupal_Exporter}} {{:Drupal_Exporter}}  + [[Drupal_Exporter|Read more... ]] [[Drupal_Exporter|Read more... ]] === b2evolution === === b2evolution === {{:B2evolution_Exporter}} {{:B2evolution_Exporter}}  + [[B2evolution_Exporter|Read more... ]] [[B2evolution_Exporter|Read more... ]] === Talk Digger === === Talk Digger === {{:Talk_Digger}} {{:Talk_Digger}}  + [[Talk_Digger|Read more... ]] [[Talk_Digger|Read more... ]] === SWAML === === SWAML === {{:SWAML}} {{:SWAML}}  + [[SWAML|Read more... ]] [[SWAML|Read more... ]] === OpenLink Data Spaces (ODS) === === OpenLink Data Spaces (ODS) === {{:OpenLink_Data_Spaces}} {{:OpenLink_Data_Spaces}}  + [[OpenLink_Data_Spaces|Read more... ]] [[OpenLink_Data_Spaces|Read more... ]] === vBulletin === === vBulletin === {{:VBSIOC}} {{:VBSIOC}}  + [[VBSIOC|Read more... ]] [[VBSIOC|Read more... ]] Line 64: Line 73: == PHP == == PHP == {{:PHPExportAPI}} {{:PHPExportAPI}}  + [[PHPExportAPI|Read more... ]] [[PHPExportAPI|Read more... ]] Line 90: Line 100: === SIOC Browser === === SIOC Browser === {{:SiocRdfBrowser}} {{:SiocRdfBrowser}}  + [[SiocRdfBrowser|Read more... ]] [[SiocRdfBrowser|Read more... ]] Line 102: Line 113: === SIOC Explorer === === SIOC Explorer === {{:SiocExplorer}} {{:SiocExplorer}}  + [[SiocExplorer|Read more... ]] [[SiocExplorer|Read more... ]] Line 108: Line 120: === Wordpress Importer=== === Wordpress Importer=== {{:SIOC_Import_Plugin}} {{:SIOC_Import_Plugin}}  + [[SIOC_Import_Plugin|Read more... ]] [[SIOC_Import_Plugin|Read more... ]] Line 113: Line 126: == Semantic Radar == == Semantic Radar == {{:SemanticRadar}} {{:SemanticRadar}}  + [[SemanticRadar|Read more... ]] [[SemanticRadar|Read more... ]]

ToDo -

←Older revision Revision as of 16:36, 2 May 2008 Line 17: Line 17: Bugfixes: Bugfixes: - * Plugin uses email_sha1 (correct) but also email_sha1sum (wrong)+* Plugin uses email_sha1 (correct) but also email_sha1sum (wrong) - * replace reply_to by reply_of in comments export+* replace reply_to by reply_of in comments export - * update the SIOC plugin according to ontology update - including changes in the role of sioc:User, foaf:Person and namespace changes for dcterms+* update the SIOC plugin according to ontology update - including: - * see what's the problem with [http://johnbreslin.com/blog/index.php?sioc_type=post&sioc_id=450 post #450]+** changes in the role of sioc:User, foaf:Person and namespace changes for dcterms - * don't show properties that have empty string value (e.g., homepage="")+** use sioct:BlogPost etc. - * doesn't work now with WPMU RC1 (JB)+* see what's the problem with [http://johnbreslin.com/blog/index.php?sioc_type=post&sioc_id=450 post #450] - * remove created from sioc:User (JB)+* don't show properties that have empty string value (e.g., homepage="")  +* doesn't work now with WPMU RC1 (JB)  +* remove created from sioc:User (JB) Improvements: Improvements: - * WordPress 2.3 - when editing a blog post I can now see who trackbacks have been sent to - expose that via a sioc:reply_of in of the WordPress plugin?+* WordPress 2.3 - when editing a blog post I can now see who trackbacks have been sent to - expose that via a sioc:reply_of in of the WordPress plugin? - * ping PTSW, ... when content is created +* ping PTSW, ... when content is created - * use PHP SIOC API+* use PHP SIOC API - * use filters to "implant" RDFa info into posts+* use filters to "implant" RDFa info into posts - * adapt to export the list of blogs from WordPress MU+* adapt to export the list of blogs from WordPress MU - * integrate SIOC WP plugin with Morten's FOAF plugin+* integrate SIOC WP plugin with Morten's FOAF plugin - * add SKOS export+* add SKOS export - * export information about tags (currently have categories)+* export information about tags (currently have categories) - * fix problem exporting content:encoded with parseType="Literal"+* fix problem exporting content:encoded with parseType="Literal" - * add auto-notification of new plugin versions - see [http://wiki.wordpress.org/Plugin Plugins] and [http://wiki.wordpress.org/AutomaticUpdate Automatic Update] (WP wiki)+* add auto-notification of new plugin versions - see [http://wiki.wordpress.org/Plugin Plugins] and [http://wiki.wordpress.org/AutomaticUpdate Automatic Update] (WP wiki) - * enrich blgo pages with RDFa seeAlso pointers to SIOC data - as [GNU] has done by hacking templates+* enrich blgo pages with RDFa seeAlso pointers to SIOC data - as [GNU] has done by hacking templates Maybe: Maybe: - * in the Post SIOC information - has_creator property add rdfs:seeAlso link to the SIOC info about the author.+* in the Post SIOC information - has_creator property add rdfs:seeAlso link to the SIOC info about the author. - * (low pri) add paging for the list of sioc:Users - as Danny had quite many sioc:Users in the SIOC export +* (low pri) add paging for the list of sioc:Users - as Danny had quite many sioc:Users in the SIOC export - * add extracting of eRDF / RDFa from blog posts+* add extracting of eRDF / RDFa from blog posts Won't fix: Won't fix: - * if original post has invalid Unicode characters - as in [http://www.johnbreslin.com/blog/2005/02/11/the-friday-interview-tom-murphy-boardsie/ (post #182) Friday Interview - Tom Murphy - boards.ie] or [http://www.johnbreslin.com/blog/2005/01/21/vicipid-wikipedia-as-gaeilge/ (post #173) Wikipedia as Gaeilge]+* if original post has invalid Unicode characters - as in [http://www.johnbreslin.com/blog/2005/02/11/the-friday-interview-tom-murphy-boardsie/ (post #182) Friday Interview - Tom Murphy - boards.ie] or [http://www.johnbreslin.com/blog/2005/01/21/vicipid-wikipedia-as-gaeilge/ (post #173) Wikipedia as Gaeilge] - * [http://johnbreslin.com/blog/index.php?sioc_type=post&sioc_id=74 not well-formed XML in post #74] - invalid control chars in the original post+* [http://johnbreslin.com/blog/index.php?sioc_type=post&sioc_id=74 not well-formed XML in post #74] - invalid control chars in the original post [[Category:Applications]] [[Category:Applications]]

←Older revision Revision as of 11:15, 2 May 2008 Line 57: Line 57: == SVN access == == SVN access == -A somewhat stable version:+A somewhat stable version using ARC: svn co http://sw.deri.org/svn/uldis/2007/05/sioc_php_import/ svn co http://sw.deri.org/svn/uldis/2007/05/sioc_php_import/ -An experimental, more advanced version:+An experimental, more advanced version using RAP: svn co http://sw.deri.org/svn/uldis/2008/04/sioc_php_import_experimental svn co http://sw.deri.org/svn/uldis/2008/04/sioc_php_import_experimental  +  += ToDos =  +  +* import categories  +  +Ideas:  +* an imported post could be somehow marked as a sioc:sibling of the original post  +* probably we shouldn't import trackbacks to old posts that are listed as comment for a imported post [[Category:Applications]] [[Category:Applications]]

i just saw that boards.ie doesn't use "similar threads" feature. i saw it on boards.us and assumed it's used here as well, but it isn't

←Older revision Revision as of 13:48, 1 May 2008 Line 5: Line 5: :''For more explanation, see [[Data/Boards.ie]].'' :''For more explanation, see [[Data/Boards.ie]].'' -The discussions on the Irish web forum site http://boards.ie/ are available as SIOC here. The top-level site documents link to users (and on to FOAF files) as well as top-level forums. Forums link to subforums and threads, which finally link to individual posts. The posts link to each other based on replying and quoting. The FOAF files link to each other, describing a social network based on the users' "buddy lists". There are also links between threads that are deemed similar.+The discussions on the Irish web forum site http://boards.ie/ are available as SIOC here. The top-level site documents link to users (and on to FOAF files) as well as top-level forums. Forums link to subforums and threads, which finally link to individual posts. The posts link to each other based on replying and quoting. The FOAF files link to each other, describing a social network based on the users' "buddy lists". The data in total over 10 years is around 9 million documents and takes about 50 gigabytes of disk space, so we slice it into smaller archives. The first slices are all data for the year 1998, and the site, forum, user and FOAF documents, all in the RDF/XML file format. The data in total over 10 years is around 9 million documents and takes about 50 gigabytes of disk space, so we slice it into smaller archives. The first slices are all data for the year 1998, and the site, forum, user and FOAF documents, all in the RDF/XML file format.

←Older revision Revision as of 15:36, 25 April 2008 Line 64: Line 64: == Ontology == == Ontology == -* allow a broader use of some properties such as sioc:note, sioc:content, sioc:sibling and sioc:num_replies by extending them to sioc:Item (they are currently restricted to sioc:Post) see http://tuukka.iki.fi/tmp/sioc-2008-02-29.html#10:42:56+* <strike>allow a broader use of some properties such as sioc:note, sioc:content, sioc:sibling and sioc:num_replies by extending them to sioc:Item (they are currently restricted to sioc:Post) see http://tuukka.iki.fi/tmp/sioc-2008-02-29.html#10:42:56</strike> -* also drop the domain of sioc:has_administrator and the range of sioc:administrator_of as they are currently set to sioc:Site but e.g. Yuki needs them also for sioc:Forum+* <strike>also drop the domain of sioc:has_administrator and the range of sioc:administrator_of as they are currently set to sioc:Site but e.g. Yuki needs them also for sioc:Forum</strike> === Follow up from last discussion === === Follow up from last discussion === Line 87: Line 87: === Dublin Core === === Dublin Core === -After the [http://dublincore.org/news/2008/#dcmi-news-20080114-01 major updated of last January], it's important to review the relation between SIOC and DC. In summary, all the RDF stuff has been moved to [http://dublincore.org/documents/2008/01/14/dcmi-terms/ dct], with [http://dublincore.org/documents/2008/01/14/domain-range/ formal domain/ranges of properties].+<strike>After the [http://dublincore.org/news/2008/#dcmi-news-20080114-01 major updated of last January], it's important to review the relation between SIOC and DC. In summary, all the RDF stuff has been moved to [http://dublincore.org/documents/2008/01/14/dcmi-terms/ dct], with [http://dublincore.org/documents/2008/01/14/domain-range/ formal domain/ranges of properties].</strike> ==== sioc:topic ==== ==== sioc:topic ====

←Older revision Revision as of 11:32, 25 April 2008 Line 1: Line 1: -=== Things common to some vocabularies that we cannot do at the moment === +== Things common to some vocabularies that we cannot do at the moment == * rdfs:Literal as rdfs:range of a property * rdfs:Literal as rdfs:range of a property Line 15: Line 15: * owl:disjointWith is used by foaf and sioc, but not by skos * owl:disjointWith is used by foaf and sioc, but not by skos -=== Vocabulary specific stuff we probably don't want to do ===+== Vocabulary specific stuff we probably don't want to do == foaf: foaf: Line 53: Line 53: </skos:changeNote> </skos:changeNote>  +== Vocabularies that we can/will be able to recreate ==  +  +=== Completely ===  +[http://64.233.183.104/search?q=cache:zTYG34a3i1cJ:dannyayers.com:88/xmlns/ibis/index.htm IBIS] - Issue-Based Information Systems by Danny Ayers  +  +  +=== Mostly ===  +[http://equiforum.org/downloads/EQUIFORUM_CRML_PROPOSAL_JUNE_2004.pdf CRML] - Conflict Resolution Markup Language by Jonathan Hendler  +  +Missing:  +* owl:unionOf  +* protege:allowedParent [[Category:Neologism|FeatureAnalysis]] [[Category:Neologism|FeatureAnalysis]]

←Older revision Revision as of 11:01, 25 April 2008 Line 9: Line 9: [[BrainStorming/2008-01-29]] [[BrainStorming/2008-01-29]] -== ToDo ==+[[BrainStorming/2008-04-09]] - + -=== now - 5d 2007.02.08 ===+ - + -todo, meetings before Alex leaves+ - + -* (any time) all - microblogging (link to the wiki page or not?)+ - + -done:+ -* all - promoting SIOC+ -** building a community+ -** information on wikipedia, http://esw.w3.org/topic/SemanticWebTools, ...+ -* all - SIOC tools development+ -** best practices for publishing SIOC+ -* 4d - SIOC / Tabulator hacking session+ -* 5d - URIs in SIOC ([[URIBestPractices]])+ -** 12:00am Knud - SIOC in conference metadata+ -** 1:30pm Richard - Social graph+ -** 3:00pm Tomas - distributed blogging (http://sw.deri.org/wiki/DistributedBlogging)+ - + -=== 1d 2007.02.11 - 5d 2007.02.15 ===+ [[Category:Meetings]] [[Category:Meetings]]

←Older revision Revision as of 12:11, 24 April 2008 Line 1: Line 1: <onlyinclude>''This information needs to be reviewed and possibly updated.'' [[Category:Review_Information]]</onlyinclude> <onlyinclude>''This information needs to be reviewed and possibly updated.'' [[Category:Review_Information]]</onlyinclude> -See "What links here" to get a list of pages that contain possibly outdated information.+   +See "What links here" to get a list of pages that contain possibly outdated information. (That should be the same as the content of Category:Review_Information)