허재욱 / ComNetPrj

Go to a project
Toggle navigation Toggle navigation pinning
Switch branch/tag
index.md 41 KB

parse5

NPM Version

WHATWG HTML5 specification-compliant, fast and ready for production HTML parsing/serialization toolset for Node.

parse5 contains nearly everything what you will need to deal with the HTML. It's the fastest spec-compliant HTML parser for Node to the date and will parse HTML the way the latest version of your browser does. It's stable and used by such projects as jsdom, Angular2, Polymer and many more.

Table of contents

Install

$ npm install parse5

Usage

var parse5 = require('parse5');

var document     = parse5.parse('<!DOCTYPE html><html><body>Hi there!</body></html>');
var documentHtml = parse5.serialize(document);


var fragment     = parse5.parseFragment('<td>Yo!</td>');
var fragmentHtml = parse5.serialize(fragment);

For more advanced examples see API reference and FAQ.

API Reference

Objects

parse5 : object

Typedefs

ElementLocationInfo : Object
LocationInfo : Object
ParserOptions : Object
SAXParserOptions : Object
SerializerOptions : Object
TreeAdapter : Object

parse5 : object

Kind: global namespace

parse5.ParserStream ⇐ stream.Writable

Kind: instance class of parse5 Extends: stream.Writable

new ParserStream(options)

Streaming HTML parser with the scripting support. Writable stream.

Param Type Description
options ParserOptions Parsing options.

Example

var parse5 = require('parse5');
var http = require('http');

// Fetch google.com content and obtain it's <body> node
http.get('http://google.com', function(res) {
 var parser = new parse5.ParserStream();

 parser.on('finish', function() {
     var body = parser.document.childNodes[0].childNodes[1];
 });

 res.pipe(parser);
});

parserStream.document : ASTNode.<document>

Resulting document node.

Kind: instance property of ParserStream

"script" (scriptElement, documentWrite(html), resume)

Raised then parser encounters <script> element. If event has listeners then parsing will be suspended on event emission. So, if <script> has src attribute you can fetch it, execute and then resume parser like browsers do.

Kind: event emitted by ParserStream

Param Type Description
scriptElement ASTNode Script element that caused the event.
documentWrite(html) function Write additional html at the current parsing position. Suitable for the DOM document.write and document.writeln methods implementation.
resume function Resumes the parser.

Example

var parse = require('parse5');
var http = require('http');

var parser = new parse5.ParserStream();

parser.on('script', function(scriptElement, documentWrite, resume) {
  var src = parse5.treeAdapters.default.getAttrList(scriptElement)[0].value;

  http.get(src, function(res) {
     // Fetch script content, execute it with DOM built around `parser.document` and
     // `document.write` implemented using `documentWrite`
     ...
     // Then resume the parser
     resume();
  });
});

parser.end('<script src="example.com/script.js"></script>');

parse5.SAXParser ⇐ stream.Transform

Kind: instance class of parse5 Extends: stream.Transform

new SAXParser(options)

Streaming SAX-style HTML parser. Transform stream (which means you can pipe through it, see example).

Param Type Description
options SAXParserOptions Parsing options.

Example

var parse5 = require('parse5');
var http = require('http');
var fs = require('fs');

var file = fs.createWriteStream('/home/google.com.html');
var parser = new SAXParser();

parser.on('text', function(text) {
 // Handle page text content
 ...
});

http.get('http://google.com', function(res) {
 // SAXParser is the Transform stream, which means you can pipe
 // through it. So you can analyze page content and e.g. save it
 // to the file at the same time:
 res.pipe(parser).pipe(file);
});

saxParser.stop()

Stops parsing. Useful if you want parser to stop consume CPU time once you've obtained desired info from input stream. Doesn't prevents piping, so data will flow through parser as usual.

Kind: instance method of SAXParser Example

var parse5 = require('parse5');
var http = require('http');
var fs = require('fs');

var file = fs.createWriteStream('/home/google.com.html');
var parser = new parse5.SAXParser();

parser.on('doctype', function(name, publicId, systemId) {
 // Process doctype info ans stop parsing
 ...
 parser.stop();
});

http.get('http://google.com', function(res) {
 // Despite the fact that parser.stop() was called whole
 // content of the page will be written to the file
 res.pipe(parser).pipe(file);
});

"startTag" (name, attributes, selfClosing, [location])

Raised then parser encounters start tag.

Kind: event emitted by SAXParser

Param Type Description
name String Tag name.
attributes String List of attributes in { key: String, value: String } form.
selfClosing Boolean Indicates if tag is self-closing.
[location] LocationInfo Start tag source code location info. Available if location info is enabled in SAXParserOptions.

"endTag" (name, [location])

Raised then parser encounters end tag.

Kind: event emitted by SAXParser

Param Type Description
name String Tag name.
[location] LocationInfo End tag source code location info. Available if location info is enabled in SAXParserOptions.

"comment" (text, [location])

Raised then parser encounters comment.

Kind: event emitted by SAXParser

Param Type Description
text String Comment text.
[location] LocationInfo Comment source code location info. Available if location info is enabled in SAXParserOptions.

"doctype" (name, publicId, systemId, [location])

Raised then parser encounters document type declaration.

Kind: event emitted by SAXParser

Param Type Description
name String Document type name.
publicId String Document type public identifier.
systemId String Document type system identifier.
[location] LocationInfo Document type declaration source code location info. Available if location info is enabled in SAXParserOptions.

"text" (text, [location])

Raised then parser encounters text content.

Kind: event emitted by SAXParser

Param Type Description
text String Text content.
[location] LocationInfo Text content code location info. Available if location info is enabled in SAXParserOptions.

parse5.SerializerStream ⇐ stream.Readable

Kind: instance class of parse5 Extends: stream.Readable

new SerializerStream(node, [options])

Streaming AST node to HTML serializer. Readable stream.

Param Type Description
node ASTNode Node to serialize.
[options] SerializerOptions Serialization options.

Example

var parse5 = require('parse5');
var fs = require('fs');

var file = fs.createWriteStream('/home/index.html');

// Serialize parsed document to the HTML and write it to file
var document = parse5.parse('<body>Who is John Galt?</body>');
var serializer = new parse5.SerializerStream(document);

serializer.pipe(file);

parse5.treeAdapters

Provides built-in tree adapters which can be used for parsing and serialization.

Kind: instance property of parse5 Properties

Name Type Description
default TreeAdapter Default tree format for parse5.
htmlparser2 TreeAdapter Quite popular htmlparser2 tree format (e.g. used by cheerio and jsdom).

Example

var parse5 = require('parse5');

// Use default tree adapter for parsing
var document = parse5.parse('<div></div>', { treeAdapter: parse5.treeAdapters.default });

// Use htmlparser2 tree adapter with SerializerStream
var serializer = new parse5.SerializerStream(node, { treeAdapter: parse5.treeAdapters.htmlparser2 });

parse5.parse(html, [options]) ⇒ ASTNode.<Document>

Parses HTML string.

Kind: instance method of parse5 Returns: ASTNode.<Document> - document

Param Type Description
html string Input HTML string.
[options] ParserOptions Parsing options.

Example

var parse5 = require('parse5');

var document = parse5.parse('<!DOCTYPE html><html><head></head><body>Hi there!</body></html>');

parse5.parseFragment([fragmentContext], html, [options]) ⇒ ASTNode.<DocumentFragment>

Parses HTML fragment.

Kind: instance method of parse5 Returns: ASTNode.<DocumentFragment> - documentFragment

Param Type Description
[fragmentContext] ASTNode Parsing context element. If specified, given fragment will be parsed as if it was set to the context element's innerHTML property.
html string Input HTML fragment string.
[options] ParserOptions Parsing options.

Example

var parse5 = require('parse5');

var documentFragment = parse5.parseFragment('<table></table>');

//Parse html fragment in context of the parsed <table> element
var trFragment = parser.parseFragment(documentFragment.childNodes[0], '<tr><td>Shake it, baby</td></tr>');

parse5.serialize(node, [options]) ⇒ String

Serializes AST node to HTML string.

Kind: instance method of parse5 Returns: String - html

Param Type Description
node ASTNode Node to serialize.
[options] SerializerOptions Serialization options.

Example

var parse5 = require('parse5');

var document = parse5.parse('<!DOCTYPE html><html><head></head><body>Hi there!</body></html>');

//Serialize document
var html = parse5.serialize(document);

//Serialize <body> element content
var bodyInnerHtml = parse5.serialize(document.childNodes[0].childNodes[1]);

ElementLocationInfo : Object

Kind: global typedef Extends: LocationInfo Properties

Name Type Description
startTag LocationInfo Element's start tag LocationInfo.
endTag LocationInfo Element's end tag LocationInfo.

LocationInfo : Object

Kind: global typedef Properties

Name Type Description
line Number One-based line index
col Number One-based column index
startOffset Number Zero-based first character index
endOffset Number Zero-based last character index

ParserOptions : Object

Kind: global typedef Properties

Name Type Default Description
locationInfo Boolean false Enables source code location information for the nodes. When enabled, each node (except root node) has __location property. In case the node is not an empty element, __location will be ElementLocationInfo object, otherwise it's LocationInfo. If element was implicitly created by the parser it's __location property will be null.
treeAdapter TreeAdapter parse5.treeAdapters.default Specifies resulting tree format.

SAXParserOptions : Object

Kind: global typedef Properties

Name Type Default Description
locationInfo Boolean false Enables source code location information for the tokens. When enabled, each token event handler will receive LocationInfo object as the last argument.

SerializerOptions : Object

Kind: global typedef Properties

Name Type Default Description
treeAdapter TreeAdapter parse5.treeAdapters.default Specifies input tree format.

TreeAdapter : Object

Kind: global typedef

TreeAdapter.createDocument() ⇒ ASTNode.<Document>

Creates document node

Kind: static method of TreeAdapter Returns: ASTNode.<Document> - document See: default implementation.

TreeAdapter.createDocumentFragment() ⇒ ASTNode.<DocumentFragment>

Creates document fragment node

Kind: static method of TreeAdapter Returns: ASTNode.<DocumentFragment> - fragment See: default implementation.

TreeAdapter.createElement(tagName, namespaceURI, attrs) ⇒ ASTNode.<Element>

Creates element node

Kind: static method of TreeAdapter Returns: ASTNode.<Element> - element See: default implementation.

Param Type Description
tagName String Tag name of the element.
namespaceURI String Namespace of the element.
attrs Array Attribute name-value pair array. Foreign attributes may contain namespace and prefix fields as well.

TreeAdapter.createElement(data) ⇒ ASTNode.<CommentNode>

Creates comment node

Kind: static method of TreeAdapter Returns: ASTNode.<CommentNode> - comment See: default implementation.

Param Type Description
data String Comment text.

TreeAdapter.setDocumentType(document, name, publicId, systemId)

Sets document type. If document already have document type node in it then name, publicId and systemId properties of the node will be updated with the provided values. Otherwise, creates new document type node with the given properties and inserts it into document.

Kind: static method of TreeAdapter See: default implementation.

Param Type Description
document ASTNode.<Document> Document node.
name String Document type name.
publicId String Document type public identifier.
systemId String Document type system identifier.

TreeAdapter.setQuirksMode(document)

Sets document quirks mode flag.

Kind: static method of TreeAdapter See: default implementation.

Param Type Description
document ASTNode.<Document> Document node.

TreeAdapter.setQuirksMode(document) ⇒ Boolean

Determines if document quirks mode flag is set.

Kind: static method of TreeAdapter See: default implementation.

Param Type Description
document ASTNode.<Document> Document node.

TreeAdapter.detachNode(node)

Removes node from it's parent.

Kind: static method of TreeAdapter See: default implementation.

Param Type Description
node ASTNode Node.

TreeAdapter.insertText(parentNode, text)

Inserts text into node. If the last child of the node is the text node then provided text will be appended to the text node content. Otherwise, inserts new text node with the given text.

Kind: static method of TreeAdapter See: default implementation.

Param Type Description
parentNode ASTNode Node to insert text into.
text String Text to insert.

TreeAdapter.insertTextBefore(parentNode, text, referenceNode)

Inserts text into node before the referenced child node. If node before the referenced child node is the text node then provided text will be appended to the text node content. Otherwise, inserts new text node with the given text before the referenced child node.

Kind: static method of TreeAdapter See: default implementation.

Param Type Description
parentNode ASTNode Node to insert text into.
text String Text to insert.
referenceNode ASTNode Node to insert text before.

TreeAdapter.adoptAttributes(recipientNode, attrs)

Copies attributes to the given node. Only those nodes which are not yet present in the node are copied.

Kind: static method of TreeAdapter See: default implementation.

Param Type Description
recipientNode ASTNode Node to copy attributes into.
attrs Array Attributes to copy.

TreeAdapter.getFirstChild(node) ⇒ ASTNode

Returns first child of the given node.

Kind: static method of TreeAdapter Returns: ASTNode - firstChild See: default implementation.

Param Type Description
node ASTNode Node.

TreeAdapter.getChildNodes(node) ⇒ Array

Returns array of the given node's children.

Kind: static method of TreeAdapter Returns: Array - children See: default implementation.

Param Type Description
node ASTNode Node.

TreeAdapter.getParentNode(node) ⇒ ASTNode

Returns given node's parent.

Kind: static method of TreeAdapter Returns: ASTNode - parent See: default implementation.

Param Type Description
node ASTNode Node.

TreeAdapter.getAttrList(node) ⇒ Array

Returns array of the given node's attributes in form of the name-value pair. Foreign attributes may contain namespace and prefix fields as well.

Kind: static method of TreeAdapter Returns: Array - attributes See: default implementation.

Param Type Description
node ASTNode Node.

TreeAdapter.getTagName(element) ⇒ String

Returns given element's tag name.

Kind: static method of TreeAdapter Returns: String - tagName See: default implementation.

Param Type Description
element ASTNode.<Element> Element.

TreeAdapter.getNamespaceURI(element) ⇒ String

Returns given element's namespace.

Kind: static method of TreeAdapter Returns: String - namespaceURI See: default implementation.

Param Type Description
element ASTNode.<Element> Element.

TreeAdapter.getTextNodeContent(textNode) ⇒ String

Returns given text node's content.

Kind: static method of TreeAdapter Returns: String - text See: default implementation.

Param Type Description
textNode ASTNode.<Text> Text node.

TreeAdapter.getTextNodeContent(commentNode) ⇒ String

Returns given comment node's content.

Kind: static method of TreeAdapter Returns: String - commentText See: default implementation.

Param Type Description
commentNode ASTNode.<Comment> Comment node.

TreeAdapter.getDocumentTypeNodeName(doctypeNode) ⇒ String

Returns given document type node's name.

Kind: static method of TreeAdapter Returns: String - name See: default implementation.

Param Type Description
doctypeNode ASTNode.<DocumentType> Document type node.

TreeAdapter.getDocumentTypeNodePublicId(doctypeNode) ⇒ String

Returns given document type node's public identifier.

Kind: static method of TreeAdapter Returns: String - publicId See: default implementation.

Param Type Description
doctypeNode ASTNode.<DocumentType> Document type node.

TreeAdapter.getDocumentTypeNodeSystemId(doctypeNode) ⇒ String

Returns given document type node's system identifier.

Kind: static method of TreeAdapter Returns: String - systemId See: default implementation.

Param Type Description
doctypeNode ASTNode.<DocumentType> Document type node.

TreeAdapter.isTextNode(node) ⇒ Boolean

Determines if given node is a text node.

Kind: static method of TreeAdapter See: default implementation.

Param Type Description
node ASTNode Node.

TreeAdapter.isCommentNode(node) ⇒ Boolean

Determines if given node is a comment node.

Kind: static method of TreeAdapter See: default implementation.

Param Type Description
node ASTNode Node.

TreeAdapter.isDocumentTypeNode(node) ⇒ Boolean

Determines if given node is a document type node.

Kind: static method of TreeAdapter See: default implementation.

Param Type Description
node ASTNode Node.

TreeAdapter.isElementNode(node) ⇒ Boolean

Determines if given node is an element.

Kind: static method of TreeAdapter See: default implementation.

Param Type Description
node ASTNode Node.

FAQ

Q: I want to work with my own document tree format. How can I achieve this?

You can create a custom tree adapter so parse5 can work with your own DOM-tree implementation. Then just pass it to the parser or serializer via option:

var parse5 = require('parse5');

var myTreeAdapter = {
   //Adapter methods...
};

var document = parse5.parse('<div></div>', { treeAdapter: myTreeAdapter });

var html = parse5.serialize(document, { treeAdapter: myTreeAdapter });

You can find description of the methods which should be exposed by tree adapter and links to their default implementation in the API reference.

Q: How can I use parse5 in the browser?

Just compile it with browserify and you're set.

Q: I'm parsing <img src="foo"> with the SAXParser and I expect selfClosing flag to be true for the <img> tag. But it's not. Is there something wrong with parser?

No. Self-closing tag is the tag that has / before the closing bracket. E.g: <br/>, <meta/>. In the provided example tag just doesn't have end tag. Self-closing tags and tags without end tags are differently treated by the parser: in case of self-closing tag parser will not lookup for the appropriate closing tag and expects element to not have any content. But if start tag is not self-closing parser will treat everything after it (with the few exceptions) as the element content. However, if the start tag is in the list of void elements parser expects corresponding element to not have content and behaves the same way as the if element is self-closing. So, semantically if element is the void element self-closing tags and tags without closing tags are equivalent, but it's not true for all other tags.

TL;DR: selfClosing is the part of the lexical information and will be set only if the tag in source code has / before the closing bracket.

Q: I have some weird output from the parser, seems like it's a bug.

More likely, it's not. There are a lot of weird edge cases in HTML5 parsing algorithm, e.g.:

<b>1<p>2</b>3</p>

will be parsed as

<b>1</b><p><b>2</b>3</p>

Just try it in the latest version of your browser before submitting the issue.

Version history

2.0.0

1.5.0

  • Add: Location info for the element start and end tags (by @sakagg).

1.4.2

  • Fix: htmlparser2 tree adapter DocumentType.data property rendering (GH #45).

1.4.1

  • Fix: Location info handling for the implicitly generated <html> and <body> elements (GH #44).

1.4.0

1.3.2

  • Fix: <form> processing in <template> (GH #40).

1.3.1

  • Fix: text node in <template> serialization problem with custom tree adapter (GH #38).

1.3.0

  • Add: Serializer encodeHtmlEntities option.

1.2.0

  • Add: <template> support
  • parseFragment now uses <template> as default contextElement. This leads to the more "forgiving" parsing manner.
  • TreeSerializer was renamed to Serializer. However, serializer is accessible as parse5.TreeSerializer for backward compatibility .

1.1.6

  • Fix: apply latest changes to the htmlparser2 tree format (DOM Level1 node emulation).

1.1.5

  • Add: jsdom-specific parser with scripting support. Undocumented for jsdom internal use only.

1.1.4

  • Add: logo
  • Fix: use fake document element for fragment parsing (required by jsdom).

1.1.3

  • Development files (e.g. .travis.yml, .editorconfig) are removed from NPM package.

1.1.2

  • Fix: crash on Linux due to upper-case leading character in module name used in require().

1.1.1

  • Add: SimpleApiParser.
  • Fix: new line serialization in <pre>.
  • Fix: SYSTEM-only DOCTYPE serialization.
  • Fix: quotes serialization in DOCTYPE IDs.

1.0.0

  • First stable release, switch to semantic versioning.

0.8.3

  • Fix: siblings calculation bug in appendChild in htmlparser2 tree adapter.

0.8.1

0.6.1

  • Fix: incorrect <menuitem> handling in <body>.

0.6.0

  • Initial release.

License

Copyright (c) 2013-2015 Ivan Nikulin (ifaaan@gmail.com, https://github.com/inikulin)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.