Method |
Description |
|
checkNumberRedirects ( string $url ) : boolean |
Check if number of redirect count isn't reach. |
|
cleanupUrl ( string $url ) : string |
Cleanup URL and retrieve the final url to be called. |
|
getMetaRefreshURL ( string $url, string $html ) : false | string |
Try to find the refresh url from the meta. |
|
getReferer ( string $url, array $httpHeader = [] ) : string |
Find a Referer for this url. |
|
getUglyURL ( string $url, string $html ) : false | string |
Some website (like Blogger) define an alternative url used by robots
so that they can crawl the website which is usually full JS. |
|
getUserAgent ( string $url, array $httpHeader = [] ) : string |
Find a UserAgent for this url. |
|
headerOnlyType ( string $contentType ) : boolean |
Look for full mime type (e.g. image/jpeg) or just type (e.g. image)
to determine if the request is a binary resource. |
|
possibleUnsupportedType ( string $url ) : string | false |
Try to determine if the url is a direct link to a binary resource
by checking the extension. |
|
sendResults ( array $data ) : array |
Return results from fetch() and also re-init static variable for the next request. |
|