PHP Class Readability\Readability

Differences between the PHP port and the original ------------------------------------------------------ Arc90's Readability is designed to run in the browser. It works on the DOM tree (the parsed HTML) after the page's CSS styles have been applied and Javascript code executed. This PHP port does not run inside a browser. We use PHP's ability to parse HTML to build our DOM tree, but we cannot rely on CSS or Javascript support. As such, the results will not always match Arc90's Readability. (For example, if a web page contains CSS style rules or Javascript code which hide certain HTML elements from display, Arc90's Readability will dismiss those from consideration but our PHP port, unable to understand CSS or Javascript, will not know any better.) Another significant difference is that the aim of Arc90's Readability is to re-present the main content block of a given web page so users can read it more easily in their browsers. Correct identification, clean up, and separation of the content block is only a part of this process. This PHP port is only concerned with this part, it does not include code that relates to presentation in the browser - Arc90 already do that extremely well, and for PDF output there's FiveFilters.org's PDF Newspaper: http://fivefilters.org/pdf-newspaper/. Finally, this class contains methods that might be useful for developers working on HTML document fragments. So without deviating too much from the original code (which I don't want to do because it makes debugging and updating more difficult), I've tried to make it a little more developer friendly. You should be able to use the methods here on existing DOMElement objects without passing an entire HTML document to be parsed.
Afficher le fichier Open project: j0k3r/php-readability Class Usage Examples

Méthodes publiques

Свойство Type Description
$articleContent
$articleTitle
$convertLinksToFootnotes
$debug no more used, keept to avoid BC
$dom
$lightClean preserves more content (experimental)
$original_html
$regexps Defined up here so we don't instantiate them repeatedly in loops.
$revertForcedParagraphElements
$tidied
$tidy_config
$url optional - URL where HTML was retrieved

Protected Properties

Свойство Type Description
$body
$bodyCache Cache the body HTML in case we need to re-use it later
$domainRegExp article domain regexp for calibration
$flags 1 | 2 | 4; // Start with all processing flags set.
$html
$logger
$parser
$post_filters output HTML filters
$pre_filters raw HTML filters
$success indicates whether we were able to extract or not
$useTidy

Méthodes publiques

Méthode Description
__construct ( $html, $url = null, $parser = 'libxml', $use_tidy = true ) Create instance of Readability.
addFlag ( integer $flag ) Add a flag.
addFootnotes ( DOMElement $articleContent ) For easier reading, convert this document to have footnotes at the bottom rather than inline links.
addPostFilter ( $filter, $replacer = '' ) Add post filter for raw output HTML processing.
addPreFilter ( $filter, $replacer = '' ) Add pre filter for raw input HTML processing.
clean ( DOMElement $e, string $tag ) Clean a node of all elements of type "tag".
cleanConditionally ( DOMElement $e, string $tag ) Clean an element of all tags of type "tag" if they look fishy.
cleanHeaders ( DOMElement $e ) Clean out spurious headers from an Element. Checks things like classnames and link density.
cleanStyles ( DOMElement $e ) Remove the style attribute on every $e and under.
flagIsActive ( integer $flag ) : boolean Check if the given flag is active.
getCommaCount ( string $text ) : integer Get comma number for a given text.
getContent ( ) : DOMElement Get article content element.
getInnerText ( DOMElement $e, boolean $normalizeSpaces = true, boolean $flattenLines = false ) : string Get the inner text of a node.
getLinkDensity ( DOMElement $e, string $excludeExternal = false ) : integer Get the density of links as a percentage of the content This is the amount of text that is inside a link divided by the total text in the node.
getTitle ( ) : DOMElement Get article title element.
getWeight ( DOMElement $e ) : integer Get an element relative weight.
getWordCount ( string $text ) : integer Get words number for a given text if words separated by a space.
init ( ) : boolean Runs readability.
killBreaks ( DOMElement $node ) Remove extraneous break tags from a node.
postProcessContent ( DOMElement $articleContent ) Run any post-process modifications to article content as necessary.
prepArticle ( DOMElement $articleContent ) Prepare the article node for display. Clean out any inline styles, iframes, forms, strip extraneous

tags, etc.

removeFlag ( integer $flag ) Remove a flag.
setLogger ( Psr\Log\LoggerInterface $logger )

Méthodes protégées

Méthode Description
dbg ( $msg ) Debug.
dump_dbg ( ) Dump debug info.
getArticleTitle ( ) : DOMElement Get the article title as an H1.
grabArticle ( DOMElement $page = null ) : DOMElement | boolean Using a variety of metrics (content score, classname, element types), find the content that is most likely to be the stuff a user wants to read. Then return it wrapped up in a div.
initializeNode ( DOMElement $node ) Initialize a node with the readability object. Also checks the className/id for special names to add to its score.
prepDocument ( ) Prepare the HTML document for readability to scrape it.
reinitBody ( ) Will recreate previously deleted body property.
weightAttribute ( DOMElement $element, string $attribute ) : integer Get an element weight by attribute.

Private Methods

Méthode Description
loadHtml ( ) Load HTML in a DOMDocument.

Method Details

__construct() public méthode

Create instance of Readability.
public __construct ( $html, $url = null, $parser = 'libxml', $use_tidy = true )

addFlag() public méthode

Add a flag.
public addFlag ( integer $flag )
$flag integer

addFootnotes() public méthode

For easier reading, convert this document to have footnotes at the bottom rather than inline links.
See also: http://www.roughtype.com/archives/2010/05/experiments_in.php
public addFootnotes ( DOMElement $articleContent )
$articleContent DOMElement

addPostFilter() public méthode

Add post filter for raw output HTML processing.
public addPostFilter ( $filter, $replacer = '' )

addPreFilter() public méthode

Add pre filter for raw input HTML processing.
public addPreFilter ( $filter, $replacer = '' )

clean() public méthode

(Unless it's a youtube/vimeo video. People love movies.). Updated 2012-09-18 to preserve youtube/vimeo iframes
public clean ( DOMElement $e, string $tag )
$e DOMElement
$tag string

cleanConditionally() public méthode

"Fishy" is an algorithm based on content length, classnames, link density, number of images & embeds, etc.
public cleanConditionally ( DOMElement $e, string $tag )
$e DOMElement
$tag string

cleanHeaders() public méthode

Clean out spurious headers from an Element. Checks things like classnames and link density.
public cleanHeaders ( DOMElement $e )
$e DOMElement

cleanStyles() public méthode

Remove the style attribute on every $e and under.
public cleanStyles ( DOMElement $e )
$e DOMElement

dbg() protected méthode

Debug.
Deprecation: use $this->logger->debug() instead
protected dbg ( $msg )

dump_dbg() protected méthode

Dump debug info.
Deprecation: since Monolog gather log, we don't need it
protected dump_dbg ( )

flagIsActive() public méthode

Check if the given flag is active.
public flagIsActive ( integer $flag ) : boolean
$flag integer
Résultat boolean

getArticleTitle() protected méthode

Get the article title as an H1.
protected getArticleTitle ( ) : DOMElement
Résultat DOMElement

getCommaCount() public méthode

Get comma number for a given text.
public getCommaCount ( string $text ) : integer
$text string
Résultat integer

getContent() public méthode

Get article content element.
public getContent ( ) : DOMElement
Résultat DOMElement

getInnerText() public méthode

This also strips out any excess whitespace to be found.
public getInnerText ( DOMElement $e, boolean $normalizeSpaces = true, boolean $flattenLines = false ) : string
$e DOMElement
$normalizeSpaces boolean (default: true)
$flattenLines boolean (default: false)
Résultat string

getLinkDensity() public méthode

Can exclude external references to differentiate between simple text and menus/infoblocks.
public getLinkDensity ( DOMElement $e, string $excludeExternal = false ) : integer
$e DOMElement
$excludeExternal string
Résultat integer

getTitle() public méthode

Get article title element.
public getTitle ( ) : DOMElement
Résultat DOMElement

getWeight() public méthode

Get an element relative weight.
public getWeight ( DOMElement $e ) : integer
$e DOMElement
Résultat integer

getWordCount() public méthode

Input string should be normalized.
public getWordCount ( string $text ) : integer
$text string
Résultat integer

grabArticle() protected méthode

Using a variety of metrics (content score, classname, element types), find the content that is most likely to be the stuff a user wants to read. Then return it wrapped up in a div.
protected grabArticle ( DOMElement $page = null ) : DOMElement | boolean
$page DOMElement
Résultat DOMElement | boolean

init() public méthode

Workflow: 1. Prep the document by removing script tags, css, etc. 2. Build readability's DOM tree. 3. Grab the article content from the current dom tree. 4. Replace the current DOM tree with the new one. 5. Read peacefully.
public init ( ) : boolean
Résultat boolean true if we found content, false otherwise

initializeNode() protected méthode

Initialize a node with the readability object. Also checks the className/id for special names to add to its score.
protected initializeNode ( DOMElement $node )
$node DOMElement

killBreaks() public méthode

Remove extraneous break tags from a node.
public killBreaks ( DOMElement $node )
$node DOMElement

postProcessContent() public méthode

Run any post-process modifications to article content as necessary.
public postProcessContent ( DOMElement $articleContent )
$articleContent DOMElement

prepArticle() public méthode

Prepare the article node for display. Clean out any inline styles, iframes, forms, strip extraneous

tags, etc.

public prepArticle ( DOMElement $articleContent )
$articleContent DOMElement

prepDocument() protected méthode

This includes things like stripping javascript, CSS, and handling terrible markup.
protected prepDocument ( )

reinitBody() protected méthode

Will recreate previously deleted body property.
protected reinitBody ( )

removeFlag() public méthode

Remove a flag.
public removeFlag ( integer $flag )
$flag integer

setLogger() public méthode

public setLogger ( Psr\Log\LoggerInterface $logger )
$logger Psr\Log\LoggerInterface

weightAttribute() protected méthode

Uses regular expressions to tell if this element looks good or bad.
protected weightAttribute ( DOMElement $element, string $attribute ) : integer
$element DOMElement
$attribute string
Résultat integer

Property Details

$articleContent public_oe property

public $articleContent

$articleTitle public_oe property

public $articleTitle

$body protected_oe property

protected $body

$bodyCache protected_oe property

Cache the body HTML in case we need to re-use it later
protected $bodyCache

$convertLinksToFootnotes public_oe property

public $convertLinksToFootnotes

$debug public_oe property

no more used, keept to avoid BC
public $debug

$dom public_oe property

public $dom

$domainRegExp protected_oe property

article domain regexp for calibration
protected $domainRegExp

$flags protected_oe property

1 | 2 | 4; // Start with all processing flags set.
protected $flags

$html protected_oe property

protected $html

$lightClean public_oe property

preserves more content (experimental)
public $lightClean

$logger protected_oe property

protected $logger

$original_html public_oe property

public $original_html

$parser protected_oe property

protected $parser

$post_filters protected_oe property

output HTML filters
protected $post_filters

$pre_filters protected_oe property

raw HTML filters
protected $pre_filters

$regexps public_oe property

Defined up here so we don't instantiate them repeatedly in loops.
public $regexps

$revertForcedParagraphElements public_oe property

public $revertForcedParagraphElements

$success protected_oe property

indicates whether we were able to extract or not
protected $success

$tidied public_oe property

public $tidied

$tidy_config public_oe property

public $tidy_config

$url public_oe property

optional - URL where HTML was retrieved
public $url

$useTidy protected_oe property

protected $useTidy