Showing posts with label xml. Show all posts
Showing posts with label xml. Show all posts

Friday, January 23, 2015

Eclipse xpath parser fails to search elements by the tag names

For those who encountered the same issue (when you try to access the element using xpath by the node name in Eclipse built-in parser) I would recommend either to remove namespace definition from the document at all or remove the default namespace.
Example:

<apprepo xmlns="asdasd">
<app name="good-app">
<appinfo>
<author name="John Smith"/>
<stores>
<store name="goo" price="1"/>
<store name="app" price="1.5"/>
</stores>
</appinfo>
</app>
<app name="another-good-app">
<appinfo>
<author name="James Smith"/>
<stores>
<store name="goo" price="2"/>
</stores>
</appinfo>
</app>
</apprepo>

Query: //appinfo returns "no matches"

Let's now change the namespace so that it is bound to some prefix (let's even not use that prefix in our document)

<apprepo xmlns:pref="asdasd">
<app name="good-app">
<appinfo>
<author name="John Smith"/>
<stores>
<store name="goo" price="1"/>
<store name="app" price="1.5"/>
</stores>
</appinfo>
</app>
<app name="another-good-app">
<appinfo>
<author name="James Smith"/>
<stores>
<store name="goo" price="2"/>
</stores>
</appinfo>
</app>
</apprepo>

Result: query returns all the appinfo nodes found

P.S. - the proper parsing happens even if you completely get rid of the namespace attribute. So Eclipse xpath parser is not working with the documents having the default namespace set.

Saturday, August 31, 2013

Yet another hint on "unbound prefix" in your programmatically composed SVG file

If you compose your SVG file programmatically you might experience the following issue. Your resulting file won't be parsed with the following exception coming from the parser:

com.larvalabs.svgandroid.SVGParseException: org.apache.harmony.xml.ExpatParser$ParseException: At line 4, column 0: unbound prefix
This exception basically says that somewhere in your SVG markup you use either the tags or the attributes fromt he namespace you didn't specify in SVG header. Examine your resulting SVG code for such the sort of problem. Fixing should help.

Thursday, July 05, 2012

How To Dump File Structure To XML. Part 3 (Final Part)

In Part1 we learned how to operate with XSD schemas and design you XML-based DSL. Then in Part 2 we learned how to build the parser and apply the business logic to owr application. We made our application capable to store the file structure in XML file.

Now we need the oposite operation. We need to restore file structure from XML file. Remember that that's only a structure, not the content, so every file will not hold a byte of information, just a name (aka record in file system).

Validating XML file with XSD schema


So, check this snippet:

public class XMLToFS {

 public static void reproduce(String sourceFile) throws SAXException, IOException, JAXBException{
  JAXBContext context = JAXBContext.newInstance(XTFSFolderRoot.class);
  Unmarshaller reader = context.createUnmarshaller();
  File schemaFile = new File("conf/fstoxml.xsd");
  File sampleFile = new File(sourceFile);
  validateAgainstSchema(schemaFile, sampleFile);
  XTFSFolderRoot rootNode = (XTFSFolderRoot) reader.unmarshal(sampleFile);
  long startTime = System.currentTimeMillis();
  new FSCreator(rootNode).create();
  System.out.println("Finished in " + String.valueOf(System.currentTimeMillis() - startTime) + " milliseconds");
  
 }

 static void validateAgainstSchema(File schema, File targetXml) throws SAXException, IOException{
  SchemaFactory schemaFactory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
  Schema referenceSchema = schemaFactory.newSchema(schema);
  Validator schemaValidator = referenceSchema.newValidator();
  schemaValidator.validate(new StreamSource(targetXml));
  System.out.println("Xml file conforms specified schema.");
 }
}

XML recursive going round

This is the main acting class to reproduce the file system from an XML file. It requires the schema designed in previous posts to parse the input content genereated by previousely described functionality. This snippet is also the example of how to validate xml file against xsd schema. As it is seen it's quite simple - just use the pattern.
After we have validated our input data, we should process the data to get the file structure. First we should create the root object from the data from file (aka unmarshall) - see line #9. Once we have the object we can pass it to the dedicated code to reporoduce the structure as the set of files on your hard drive. Such the functioality I incapsulated in FSCreator class. Here it is:


public class FSCreator {

 XTFSFolderRoot rootNode;
 
 public FSCreator(XTFSFolderRoot rootNode){
  this.rootNode = rootNode;
 }
 
 public void create() throws IOException{
  File rootFolder = new File(rootNode.getPath());
  if(!rootFolder.isDirectory()){
   boolean isSuccess = rootFolder.mkdir();
   if(!isSuccess){
    throw new IOException("Root folder failed to get created");
   }
  }
  reproduceFileFolderStructure(rootNode.getFolderOrFile(), rootFolder);
 }
 
 private void reproduceFileFolderStructure(List<XTFSFile> currentInput, File parent){
  for(XTFSFile item: currentInput){
   if(item instanceof XTFSFolder){
    System.out.println("Processing folder " + parent.getAbsolutePath() + "\\" + item.getName());
    File newParent = new File(parent, item.getName());
    newParent.mkdir();
    reproduceFileFolderStructure(((XTFSFolder) item).getFolderOrFile(), newParent);
   }else if(item instanceof XTFSFile){
    System.out.println("Processing file " + parent.getAbsolutePath() + "\\" + item.getName());
    try {
     new File(parent, item.getName()).createNewFile();
    } catch (IOException e) {
     System.out.println("Skipping due to IO error");
    }
   }else{
    throw new IllegalStateException("Unrecognized class of the object");
   }
  }
 }
 
}


As you can see it uses recursive function to walk through a tree and process every node depending on its type. Point the attention to lines #22, #27. We have to check which type exactly the node implements as we process files and folders in different way. It is also very important how you order type check (it's important to check if the object is of Folder type first).

The new and finalized application sources you can find here. It now has combined dump/reproduce features with single entry point. You also may download the built solution which is ready to use.

Wednesday, June 27, 2012

How To Dump File Structure To XML. Part 2.

So, here in Part 1 of the post we prepared XSD schema keeping the data-model of our data we're going to store. Now we're ready to prepare parser, wrap it with some 'business-code' and try out.

Generating JAXB parser classes

Let's do that. First of all locate your JRE (I'm sure you have one). There in /bin folder you should find xjc executable. Copy your schema there (let its name be fstoxml.xsd) and run the command 'xjc fstoxml.xsd'. What's it done? It converted the types described in your schema to the parser classes and put it to the package generated basing on the schema namespace. Check the previous part of the post to see the namespace. It is "http://www.notifymeplease.org/fstoxml/schema" which means the package will be "org.notifymeplease.fstoxml.schema". Refer to xjc help to know how to place the classes to different package.

After xjc finishes processing you find the classes under the folders corresponding to the mentioned package. These are your XML parser. Add them to sources of your Java project and get ready to write business wrapper. One important thing to note: after you have added generated classes to your sources go to XTFSFolderRoot.java and add @XmlRootElement(name = "root") annotation just before class definition. That will indicate that the element is devoted to be the root one.

Dump File Structure To XML

Remind that our business-need is to scan the file-structure (starting from some root) and dump it to XML file, so we need the recursive procedure to walk through the tree and the approach to save it. At the bottom of the page you may find the simple code with the comments describing everything that happens there.

The code fetches the value of root property from the resource bundle. The thing to know about resource bundles is that
- the file extension should always be 'properties'
- if no language code specified in the property file name as the suffix, the default one will be used.
- property file should be placed somewhere in classpath
So for current example we should have conf.properties file under the classpath

Then code creates 'root' object and fills it with the help of recursive procedure that goes across the underlying file system and populates the object dependencies.

P.S. - You may find the project sources here.
P.P.S. - You may find the built application here.
P.P.P.S. - Here is the Part 3 where I'm telling how to reproduce file system from generated XML file
To use the built application you should do the following

1. Unpack the archive to somewhere
2. Make sure you have JRE installed and JRE bin folder in your environment PATH variable
3. In the folder you've unpacked the archive to set up the root folder to scan in conf.properties
4. Run the command 'java -jar fstoxml.jar'. You should now see the generated file. For example the generated file for this project src folder looks like
.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<xtfsFolderRoot xmlns="http://www.notifymeplease.org/fstoxml/schema" path="C:\FSToXML\src">
    <folder name="org">
        <folder name="notifymeplease">
            <folder name="fstoxml">
                <file name="FSToXML.java"/>
                <folder name="schema">
                    <file name="ObjectFactory.java"/>
                    <file name="package-info.java"/>
                    <file name="XTFSFile.java"/>
                    <file name="XTFSFolder.java"/>
                    <file name="XTFSFolderRoot.java"/>
                </folder>
            </folder>
        </folder>
    </folder>
</xtfsFolderRoot>

.

Code Snippet.

This is the main code utilizing the parser.

package org.notifymeplease.fstoxml;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.ResourceBundle;

import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Marshaller;

import org.notifymeplease.fstoxml.schema.XTFSFile;
import org.notifymeplease.fstoxml.schema.XTFSFolder;
import org.notifymeplease.fstoxml.schema.XTFSFolderRoot;

public class FSToXML {

 public static void main(String[] arg) {

  ResourceBundle bundle = ResourceBundle.getBundle("conf");
  File root = new File(bundle.getString("root"));

  try {
   // Creating JAXBContext to handle the classes from specified package
   JAXBContext jaxbContext = JAXBContext.newInstance("org.notifymeplease.fstoxml.schema");
   // Create marshaller to save data to disk
   Marshaller marshaller = jaxbContext.createMarshaller();
   marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, new Boolean(true));
   // We should not start from not a directory
   if (!root.isDirectory()) {
    throw new IllegalStateException(
      "Root should represent a folder. Current root: " + root.getCanonicalPath());
   }
   // Creating the main Node. The root of further xml output
   XTFSFolderRoot rootNode = new XTFSFolderRoot();
   // Specify attribute
   rootNode.setPath(root.getCanonicalPath());
   // Populate the content
   long start = System.currentTimeMillis();
   rootNode.getFolderOrFile().addAll(populateBody(root));
   // Save to disk
   marshaller.marshal(rootNode, new FileOutputStream("jaxbOutput.xml"));
   System.out.println("Time spent(ms): " + String.valueOf(System.currentTimeMillis() - start));
  } catch (IOException e) {
   e.printStackTrace();
  } catch (JAXBException e) {
   e.printStackTrace();
  }
 }
 
 /**
  * Recursive method of populating the list 
  * @param root
  * @return
  */
 
 public static List<XTFSFile> populateBody(File root){
  File[] newSet = root.listFiles();
  ArrayList<XTFSFile> newList = new ArrayList<XTFSFile>();
  
  // Having the sequential file (which actually may appear to be a folder)
  for(File item: newSet){
   // Check if it is a file
   if(!item.isDirectory()){
    /**
     * It's a file, so just create new object,
     * set the corresponding name
     * and add it to the list
     */
    XTFSFile newFile = new XTFSFile();
    newFile.setName(item.getName());
    newList.add(newFile);
   }else{
    /**
     * It's a folder. so create folder object,
     * set the name of the folder, and populate the content
     * using the same method we're in now
     */
    XTFSFolder newFolder = new XTFSFolder();
    newFolder.setName(item.getName());
    newFolder.getFolderOrFile().addAll(populateBody(item));
    newList.add(newFolder);
   }
  }
  
  // After all has finished, return the result
  return newList;
 }

}

.

How To Dump File Structure To XML. Part 1.

Goal: Having XSD grammar to describe file structure

Assume you want to reproduce (replicate) the file structure of some host you have no the direct access to. Assume you just want to save the file structure to perform some evaluation against it. The best way is to store it in XML file. This post is about how to do that. Also as a side effect the following topics're going to be slightly touched:

- JAXB: How to generate xml file from scratch
- How to apply XSD to build convenient XML parser via xjc.

So first of all we need to design the schema of the data we're going to keep. Having such the schema will allow us to build the parser which will be used to write the data to xml and to read them from it in a very simple way. One may develop the schema by using either DTD or XSD specifications. The second one allows to use xjc translator from j2SE distribution to translate the schema to parser classes.

What meta information do we need?

So let's consider the data structure we're going to process. This should represent the minimal meta-information of the file system. So it should support having the root node, files and folders which in turn should allow holding files and folders inside. Also the structural elements (files and folders) should have the attributes like "name" (mandatory one), "isHidden", "isArchived" etc.

XSD is the format of describing the rules certain XML should be built with. It's a sort of grammar for a language which extends XML. XSD is also the XML so it can represent hierarchical structure where elements have the types dictating the way how one can (and allowed to) use them in their document meeting the schema. For example you cannot assign literal value to the attribute if you specified numeric type for it in the schema.

XSD is the declarative language so that you describe the types (which btw can extend other ones), the relationships, restrictions etc. and then use this information depending on your current need (recall that our need is to build the parser)

Such the declarative nature allows to process XSD with dedicated processors (aka translators) which can generate Java classes representing the types, dependencies and relationships of the entities described in the schema. Such the classes are marked with special annotations and getter methods so that they are easy to use to parse XML via JAXB (JavaXML Binding) technology.

Let's check an example

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" targetNamespace="http://www.notifymeplease.org/fstoxml/schema"
 xmlns:tns="http://www.notifymeplease.org/fstoxml/schema" elementFormDefault="qualified">

 <xs:element name="root" type="tns:XTFSFolderRoot"/>
 
 <xs:complexType name="XTFSFolderRoot">
  <xs:sequence minOccurs="0" maxOccurs="unbounded">
   <xs:choice>
    <xs:element name="folder" type="tns:XTFSFolder"/>
    <xs:element name="file" type="tns:XTFSFile"/>
   </xs:choice>
  </xs:sequence>
  <xs:attribute name="path" use="required"/>
 </xs:complexType>
 
 <xs:complexType name="XTFSFolder">
  <xs:complexContent>
   <xs:extension base="tns:XTFSFile">
    <xs:sequence minOccurs="0" maxOccurs="unbounded">
     <xs:choice>
      <xs:element name="folder" type="tns:XTFSFolder"/>
      <xs:element name="file" type="tns:XTFSFile"/>
     </xs:choice>
    </xs:sequence>
   </xs:extension>
  </xs:complexContent>
 </xs:complexType>

 <xs:complexType name="XTFSFile">
  <xs:attribute name="name" type="xs:string" use="required" />
  <xs:attribute name="isHidden" type="xs:boolean" use="optional"
   default="false" />
  <xs:attribute name="isReadOnly" type="xs:boolean" use="optional"
   default="false" />
  <xs:attribute name="isArchived" type="xs:boolean" use="optional"
   default="false" />
 </xs:complexType>

</xs:schema>

Mapping "types" to "tags"

This simple schema represents the data structure we're going to use. It consists of three types description: XTFSFile, XTFSFolder (which extends XTFSFile) and XTFSFolderRoot. These types will be then translated to corresponding Java classes and will contain getter methods for each the attribute returning the values of specified types for those attributes.

Note that the types do not mean tags in your xml which you're going to build according to this schema. To describe the rules of how to arrange the tags you should come up with so called "elements". Here in the example schema it is seen that we have element "root" (which describes how to use root tag in your XML). This element is of type XTFSFolderRoot. Notice that when we refer to this type we have to specify the namespace of the document we're currently developing. This is required to distinguish your custom types from the types of "http://www.w3.org/2001/XMLSchema" namespace. 

XTFSFolderRoot implies that we can have the sequence of  "folder" or "file" tags under the "root" one. This sequence can be infinite or it can not exist at all (minOccurs="0" maxOccurs="unbounded"). Folder and file tags should meet the rules described in XTFSFolder and XTFSFile types correspondingly. Note that folder type implies holding elements of either files or folders which is actually the  recursive dependency.

Types also contain the attribute description with flags of whether they are required or not, and if not then which value to use by default if they are not specified in the document.

So since we have this schema we can create some sample xml file and validate it to ensure the schema works correctly and catches all the places where the document doesn't meet it. However we are not going to use the schema to read the files but rather to write ones. Anyway we should translate it to the Java classes, include them to classpath and write the code to utilize them. 

All this stuff in the Part 2 of this post.

Monday, June 04, 2012

Good Practice: How to store the configuration of your Java-based testing framework. Part I.

Some day your framework (or any other somehow complicated project) will become so big that it will not be convenient to keep the configuration hard-coded or even in the property files. I'm probably saying not even about the configuration in regular meaning but rather about project-specific meta-information. 

What values are we looking for...

You will probably need to store:

  • Credentials
  • Some steps to perform on some stage
  • Resources and some extra attributes for them
  • Connection settings for set of databases
  • Rules for parsing certain data
  • etc.
All this stuff requires certain capabilities from your configuration storage. If you'd like to have easy usage you'd probably expect the following features from that:
  • Nice to keep all such the information in the minimum of files
  • Nice if your storage supports structural data
  • Nice if you have convenient and effective facility to read and write data out of/in that storage
  • Nice if such the format is highly standardized
  • Nice if such the format has the facility to validate the stored data for at least syntax.
  • Great if such the format won't require extra effort for infrastructure setting-up from you
  • Great if the storage keeps the data in the format which can be understood by the human
  • Great if the format allows applying of different IDEs to introduce changes in stored data manually

XML - nice alternative

Well if to take a look onto the items above the two ideas comes on my mind. First one - the idea which addresses all the Nice items. This idea is certainly relational database. However it is hard and heavy solution and may cause pretty annoying problems related to the connectivity. 
Another way is both Nice and Great. I'm talking about storing your meta-information in xml file. If you're going to use xml in your project you would probably like to apply good practice of such the pattern.

If you decided to use xml to store your data it's great to have the understanding of which exactly data you're going to store there. The very good way to understand it and to keep acting according that understanding is to design the xml-schema which does actually represent the model of data you're going to store and use.
Once you have such the schema you'll be able not to understand and troubleshoot any data-structure-related problems of your project but also will be able to use such the sweet features like validating your files against that schema and using dedicated IDEs so that you'll have the code assistance feature enabled.

Xml-schema is the sort of grammar constraints of the language your going to describe your project's meta-information with. Applying such the grammar to your configuration file will allow to catch wide set of improper data problems until your main code gets executed. 

Here you may find small but effective tutorial on how to understand and to use the schema file.

Later on I'll tell how to use all the features I told above for the real example.