PHP Class ElasticSource

Author: David Kullmann
Inheritance: extends DataSource
Afficher le fichier Open project: dkullmann/cakephp-elastic-search-datasource Class Usage Examples

Méthodes publiques

Свойство Type Description
$Http object Holds HttpSocket Object
$columns array Elasticsearch column definition
$fullDebug string Prevents error from being thrown in CakeTestFixture
$useNestedTransactions string Prevents error from being thrown in CakeTestFixture

Protected Properties

Свойство Type Description
$_document array The document for this transaction (for saving more than one model)
$_filterOps string Valid operators from DboSource to support
$_id string If we are in a transaction this is the id for the model we are indexing
$_listSources boolean Don't list sources by default, there's still some problems here
$_parent string Holding data for _parent.
$_queryLog string Query log
$_schema array Holds the mappings
$_transactionStarted boolean Track if we are in a transaction or not - allows saving multiple models to a document
$_type string If we are in a transaction this is the 'type' for the model indexing

Méthodes publiques

Méthode Description
__call ( string $method, array $arguments ) : mixed Call HttpSocket methods
__construct ( array $config = [] ) Constructor, call the parent and setup Http
addToDocument ( Model $Model, array $document = [] ) : boolean Store a document to save later, merge it with existing model information for that document
afterParseConditions ( Model $Model, array $filters = [] ) : array Perform this check after parseConditions has completed, since parseConditions is recursive we have to perform this check in a separate method (or use a static variable or whatever, but, I think this is cleaner)
allDocuments ( ) : array Get all documents for this transaction
begin ( ) : true Begin a transaction - allows saving of multiple models pseudo-automically through _bulk API, but it's main purpose is to allow saving associated model data and multiple models (saveMany/saveAssociated)
bulkIndex ( string $type = null, array $documents = [] ) : boolean Bulk index a group of models
calculate ( Model $Model, $func, $params = null )
checkMapping ( Model $Model ) : boolean Check to see if a mapping exists
commit ( ) : boolean Save all models we have been storing
create ( Model $Model, array $fields = [], array $values = [] ) : mixed If this operation is atomic then store the model data to be created on "commit" later, this allows for saving multiple models in one document through saveAll, saveMany, and saveAssociated.
createIndex ( $index, $alias = false, $settings = [] )
createSchema ( $Schema ) : void Create schema, used in testing
currentDocument ( ) : array Get the current document
currentModel ( Model &$Model ) : Model Get/set the current model - called when requests are starting
delete ( Model $Model, array $conditions = null ) : boolean Delete a record
describe ( $model ) : array Describe a model based on it's mapping
dropIndex ( $index )
dropMapping ( Model $Model ) : boolean Delete a mapping
dropSchema ( string $Schema ) : void Drop a schema
execute ( $method = null, $type = null, $api = null, $data = [] )
filterResults ( array $results = [] ) : array Filter results from a call, parsing out the records
fullTableName ( Model | string $model, boolean $quote = true, boolean $schema = true ) : string Gets full table name including prefix
generateQuery ( Model $Model, array $queryData = [] ) : array Generate an ElasticSearch query from Cake's ORM $queryData
geo ( $key, $operator, $value )
geoBoundary ( $field, array $boundaries ) : array Creates filter for a geo boundary box
geoDistance ( $field, array $options ) : array Creates filter for a geo distance search
geoDistanceRange ( $field, $minDistance ) : array Creates filter for a geo distance range search
getColumnType ( Model $model, string $column ) : string Returns the column type of a column in the model.
getLog ( ) : void Get the query log - support for DebugKit Toolbar
getMapping ( ) : array Get the entire index mapping
getType ( Model $Model ) : void Get the useType if its set, otherwise use the table name
inTransaction ( ) : void Check to see if we are in a transacion
index ( $type, $id = null, $document = [] )
insertMulti ( $type, array $fields, array $values, $refresh = true ) : void Used by CakeTestFixture to insert many records.
isConnected ( ) : boolean Returns true to signal that the source is connected to the datastore
listSources ( $data = NULL ) : array List the types available for each index
logQuery ( string $method, string $uri, string $body, string $results = [] ) : void Log a new query if debug is on
mapModel ( Model $Model, array $description = [], $alias = true ) : boolean Map a model based on it's description from MySQL (or your own)
missing ( $key, $value )
name ( string $alias ) : string Make Elastic play nice with Model::escapeField();
parseConditions ( Model $Model, array $conditions = [] ) : array Parse the 'conditions' key of a query from CakePHP's ORM
parseMapping ( array $mapping = [], $sourcesOnly = false ) : array Parse an entire index mapping to create the schema for this datasource
parseOrder ( Model $Model, $query = [] )
parseQueryType ( $query )
query ( $method, $params, Model $Model )
range ( $key, $operator, $value )
read ( Model $Model, array $queryData = [], $recursive = NULL ) : mixed Query ElasticSearch to retrieve records
reindex ( ElasticScroll $scroll, array $options = [] ) : void Copies all documents from an index type to another
reset ( ) : void Reset our transaction state, document, id, and type
resetSequence ( string $table, string $column ) : boolean Reset a sequence based on the MAX() value of $column. Useful for resetting sequences after using insertMulti().
rollback ( ) : void Cancel this save operation/transaction
scan ( Model $model, integer $pageSize = 50, string $cursorTtl = '2m' ) : ElasticScroll Returns an iterator with all results for an index type, the results order is maintained by Elastic search for $cursorTtl time in minutes. This function is commonly used for re-indexing a type when it's internal pproperty definition change.
term ( $key, $operator, $value )
truncate ( string $type, $refresh = true ) : void Used by CakeTestFixture to truncate a type
typeToAlias ( $type ) : string Convert a pluralized table name (test_models) to an alias (TestModel)
update ( Model $Model, $fields = [], $values = [], $conditions = NULL )

Méthodes protégées

Méthode Description
_addLog ( string $data = [] ) : void Added an item to the query log
_convertAttributes ( string $attr, string $val ) : string Convert MySQL or CakePHP ORM field attributes into ElasticSearch compatible attributes
_findKey ( Model $Model, array $document = [] ) : mixed Find the key for this document
_normalizeBoolFilter ( array $filter ) : array Restructures bool filters so they have the propper formatting for Elastic Search
_parseDescription ( array $description = [], $Model = null ) : array Recursive method to map a SQL-like model description into a ElasticSearch one
_parseKey ( Model $Model, string $key, mixed $value ) : array Used to parse a key for ElasticSearch filters
_parseResponse ( CakeResponse $response ) : mixed Parse the response from ElasticSearch, throwing errors if necessary
_setupTransaction ( Model $Model, array $document = [] ) : void If we are in a new transaction, set _type and _id, if we are in an existing transaction then start a new document and set _id
_throwError ( string $info ) : void Throw the right error
_uri ( array $config ) : array Get the URI for a request

Method Details

__call() public méthode

Call HttpSocket methods
See also: ElasticSource::_parseResponse()
See also: ElasticSource::_filterResults()
Author: David Kullmann
public __call ( string $method, array $arguments ) : mixed
$method string
$arguments array
Résultat mixed Results from HttpSocket calls parsed by _parseResponse and _filterResults

__construct() public méthode

Constructor, call the parent and setup Http
Author: David Kullmann
public __construct ( array $config = [] )
$config array

_addLog() protected méthode

Added an item to the query log
Author: David Kullmann
protected _addLog ( string $data = [] ) : void
$data string
Résultat void

_convertAttributes() protected méthode

Convert MySQL or CakePHP ORM field attributes into ElasticSearch compatible attributes
Author: David Kullmann
protected _convertAttributes ( string $attr, string $val ) : string
$attr string
$val string
Résultat string ES compatible attribute

_findKey() protected méthode

Find the key for this document
Author: David Kullmann
protected _findKey ( Model $Model, array $document = [] ) : mixed
$Model Model
$document array
Résultat mixed Boolean false if no key is present, otherwise the key (string/int)

_normalizeBoolFilter() protected méthode

Restructures bool filters so they have the propper formatting for Elastic Search
protected _normalizeBoolFilter ( array $filter ) : array
$filter array single bool filter created by _parseKey
Résultat array

_parseDescription() protected méthode

Recursive method to map a SQL-like model description into a ElasticSearch one
Author: David Kullmann
protected _parseDescription ( array $description = [], $Model = null ) : array
$description array
Résultat array Array representing ES Mapping

_parseKey() protected méthode

Used to parse a key for ElasticSearch filters
Author: David Kullmann
protected _parseKey ( Model $Model, string $key, mixed $value ) : array
$Model Model
$key string
$value mixed
Résultat array ElasticSearch compatible filter

_parseResponse() protected méthode

Parse the response from ElasticSearch, throwing errors if necessary
Author: David Kullmann
protected _parseResponse ( CakeResponse $response ) : mixed
$response CakeResponse
Résultat mixed boolean true or false, or body of request as array

_setupTransaction() protected méthode

If we are in a new transaction, set _type and _id, if we are in an existing transaction then start a new document and set _id
Author: David Kullmann
protected _setupTransaction ( Model $Model, array $document = [] ) : void
$Model Model
$document array
Résultat void

_throwError() protected méthode

Throw the right error
Author: David Kullmann
protected _throwError ( string $info ) : void
$info string
Résultat void

_uri() protected méthode

Get the URI for a request
Author: David Kullmann
protected _uri ( array $config ) : array
$config array HttpSocket $config style array
Résultat array Array compatible with HttpSocket $uri for get/post/put/delete

addToDocument() public méthode

Allows saveAssociated to work and allows storing multiple models in one document
Author: David Kullmann
public addToDocument ( Model $Model, array $document = [] ) : boolean
$Model Model
$document array
Résultat boolean true

afterParseConditions() public méthode

Perform this check after parseConditions has completed, since parseConditions is recursive we have to perform this check in a separate method (or use a static variable or whatever, but, I think this is cleaner)
Author: David Kullmann
public afterParseConditions ( Model $Model, array $filters = [] ) : array
$Model Model
$filters array
Résultat array

allDocuments() public méthode

Get all documents for this transaction
Author: David Kullmann
public allDocuments ( ) : array
Résultat array Array of documents keyed by their primary key

begin() public méthode

Begin a transaction - allows saving of multiple models pseudo-automically through _bulk API, but it's main purpose is to allow saving associated model data and multiple models (saveMany/saveAssociated)
Author: David Kullmann
public begin ( ) : true
Résultat true

bulkIndex() public méthode

Bulk index a group of models
Author: David Kullmann
public bulkIndex ( string $type = null, array $documents = [] ) : boolean
$type string
$documents array
Résultat boolean true on success

calculate() public méthode

public calculate ( Model $Model, $func, $params = null )
$Model Model

checkMapping() public méthode

Check to see if a mapping exists
Author: David Kullmann
public checkMapping ( Model $Model ) : boolean
$Model Model
Résultat boolean true if it exists

commit() public méthode

Save all models we have been storing
Author: David Kullmann
public commit ( ) : boolean
Résultat boolean true on success

create() public méthode

The first model to be saved in the transaction determines the type, and if more than one set of models (saveMany) are being saved then they will be stored and saved in bulk Otherwise simply index the model data given based on it's type and Id
Author: David Kullmann
public create ( Model $Model, array $fields = [], array $values = [] ) : mixed
$Model Model
$fields array
$values array
Résultat mixed boolean or array of model data

createIndex() public méthode

public createIndex ( $index, $alias = false, $settings = [] )

createSchema() public méthode

Create schema, used in testing
Author: David Kullmann
public createSchema ( $Schema ) : void
Résultat void

currentDocument() public méthode

Get the current document
Author: David Kullmann
public currentDocument ( ) : array
Résultat array Document data

currentModel() public méthode

Get/set the current model - called when requests are starting
Author: David Kullmann
public currentModel ( Model &$Model ) : Model
$Model Model
Résultat Model the current model

delete() public méthode

Delete a record
Author: David Kullmann
public delete ( Model $Model, array $conditions = null ) : boolean
$Model Model
$conditions array
Résultat boolean True on success

describe() public méthode

Describe a model based on it's mapping
Author: David Kullmann
public describe ( $model ) : array
Résultat array Schema

dropIndex() public méthode

public dropIndex ( $index )

dropMapping() public méthode

Delete a mapping
Author: David Kullmann
public dropMapping ( Model $Model ) : boolean
$Model Model
Résultat boolean true on success

dropSchema() public méthode

Drop a schema
Author: David Kullmann
public dropSchema ( string $Schema ) : void
$Schema string
Résultat void

execute() public méthode

public execute ( $method = null, $type = null, $api = null, $data = [] )

filterResults() public méthode

Filter results from a call, parsing out the records
Author: David Kullmann
public filterResults ( array $results = [] ) : array
$results array
Résultat array Array of results

fullTableName() public méthode

Gets full table name including prefix
public fullTableName ( Model | string $model, boolean $quote = true, boolean $schema = true ) : string
$model Model | string Either a Model object or a string table name.
$quote boolean Whether you want the table name quoted.
$schema boolean Whether you want the schema name included.
Résultat string Full quoted table name

generateQuery() public méthode

Generate an ElasticSearch query from Cake's ORM $queryData
Author: David Kullmann
public generateQuery ( Model $Model, array $queryData = [] ) : array
$Model Model
$queryData array
Résultat array Array that can be converted to JSON for ElasticSearch

geo() public méthode

public geo ( $key, $operator, $value )

geoBoundary() public méthode

Creates filter for a geo boundary box
public geoBoundary ( $field, array $boundaries ) : array
$boundaries array
Résultat array

geoDistance() public méthode

Creates filter for a geo distance search
public geoDistance ( $field, array $options ) : array
$options array
Résultat array

geoDistanceRange() public méthode

Creates filter for a geo distance range search
public geoDistanceRange ( $field, $minDistance ) : array
Résultat array

getColumnType() public méthode

according to its mapping.
public getColumnType ( Model $model, string $column ) : string
$model Model instance to be inspected
$column string The name of the model column
Résultat string Column type

getLog() public méthode

Get the query log - support for DebugKit Toolbar
Author: David Kullmann
public getLog ( ) : void
Résultat void

getMapping() public méthode

Get the entire index mapping
Author: David Kullmann
public getMapping ( ) : array
Résultat array ES Mapping

getType() public méthode

Get the useType if its set, otherwise use the table name
Author: David Kullmann
public getType ( Model $Model ) : void
$Model Model
Résultat void

inTransaction() public méthode

Check to see if we are in a transacion
Author: David Kullmann
public inTransaction ( ) : void
Résultat void

index() public méthode

public index ( $type, $id = null, $document = [] )

insertMulti() public méthode

You should use index instead
Author: David Kullmann
public insertMulti ( $type, array $fields, array $values, $refresh = true ) : void
$fields array
$values array
Résultat void

isConnected() public méthode

Returns true to signal that the source is connected to the datastore
public isConnected ( ) : boolean
Résultat boolean True

listSources() public méthode

List the types available for each index
Author: David Kullmann
public listSources ( $data = NULL ) : array
Résultat array Array of types - similar to tables in a DB

logQuery() public méthode

Log a new query if debug is on
Author: David Kullmann
public logQuery ( string $method, string $uri, string $body, string $results = [] ) : void
$method string
$uri string
$body string
$results string
Résultat void

mapModel() public méthode

Map a model based on it's description from MySQL (or your own)
Author: David Kullmann
public mapModel ( Model $Model, array $description = [], $alias = true ) : boolean
$Model Model
$description array
Résultat boolean true on success

missing() public méthode

public missing ( $key, $value )

name() public méthode

Make Elastic play nice with Model::escapeField();
public name ( string $alias ) : string
$alias string The model alias
Résultat string

parseConditions() public méthode

Parse the 'conditions' key of a query from CakePHP's ORM
Author: David Kullmann
public parseConditions ( Model $Model, array $conditions = [] ) : array
$Model Model
$conditions array
Résultat array Array of filters for ElasticSearch

parseMapping() public méthode

Parse an entire index mapping to create the schema for this datasource
Author: David Kullmann
public parseMapping ( array $mapping = [], $sourcesOnly = false ) : array
$mapping array
Résultat array CakePHP schema

parseOrder() public méthode

public parseOrder ( Model $Model, $query = [] )
$Model Model

parseQueryType() public méthode

public parseQueryType ( $query )

query() public méthode

public query ( $method, $params, Model $Model )
$Model Model

range() public méthode

public range ( $key, $operator, $value )

read() public méthode

Query ElasticSearch to retrieve records
Author: David Kullmann
public read ( Model $Model, array $queryData = [], $recursive = NULL ) : mixed
$Model Model
$queryData array
Résultat mixed boolean false on failure or array of records on success

reindex() public méthode

Copies all documents from an index type to another
public reindex ( ElasticScroll $scroll, array $options = [] ) : void
$scroll ElasticScroll Iterator with results to be copied, can be from another server.
$options array Array with following options: - toIndex: Target index name. If none provided then default configured index for this datasource will be used leave blanck wehn oyu want to copy data from one server to the other using same index name. - toType: Type name to use for storing new documents in target index (required) - transform: A callback function that will get each document before it is stored in target index. useful for adding, removing or changing any data before it is saved.
Résultat void

reset() public méthode

Reset our transaction state, document, id, and type
Author: David Kullmann
public reset ( ) : void
Résultat void

resetSequence() public méthode

This method should be implemented by datasources that require sequences to be used.
public resetSequence ( string $table, string $column ) : boolean
$table string The name of the table to update.
$column string The column to use when reseting the sequence value.
Résultat boolean success.

rollback() public méthode

Cancel this save operation/transaction
Author: David Kullmann
public rollback ( ) : void
Résultat void

scan() public méthode

Returns an iterator with all results for an index type, the results order is maintained by Elastic search for $cursorTtl time in minutes. This function is commonly used for re-indexing a type when it's internal pproperty definition change.
public scan ( Model $model, integer $pageSize = 50, string $cursorTtl = '2m' ) : ElasticScroll
$model Model The model intance to introspect to get the scrollable results
$pageSize integer as results are iterated, how large should the page be when asking Elastic Searhc for results bigger numbers make fewer requests to ElasticSearch but consume more memory and take longer to process in php
$cursorTtl string Time to keep cursor results cached in Elastic Seaach (Example: '10m' for ten minutes)
Résultat ElasticScroll result iterator with all entries for an index type.

term() public méthode

public term ( $key, $operator, $value )

truncate() public méthode

Used by CakeTestFixture to truncate a type
Author: David Kullmann
public truncate ( string $type, $refresh = true ) : void
$type string
Résultat void

typeToAlias() public méthode

Convert a pluralized table name (test_models) to an alias (TestModel)
Author: David Kullmann
public typeToAlias ( $type ) : string
Résultat string

update() public méthode

public update ( Model $Model, $fields = [], $values = [], $conditions = NULL )
$Model Model

Property Details

$Http public_oe property

Holds HttpSocket Object
public object $Http
Résultat object

$_document protected_oe property

The document for this transaction (for saving more than one model)
protected array $_document
Résultat array

$_filterOps protected_oe property

Valid operators from DboSource to support
protected string $_filterOps
Résultat string

$_id protected_oe property

If we are in a transaction this is the id for the model we are indexing
protected string $_id
Résultat string

$_listSources protected_oe property

Don't list sources by default, there's still some problems here
protected bool $_listSources
Résultat boolean

$_parent protected_oe property

Holding data for _parent.
protected string $_parent
Résultat string

$_queryLog protected_oe property

Query log
protected string $_queryLog
Résultat string

$_schema protected_oe property

Holds the mappings
protected array $_schema
Résultat array

$_transactionStarted protected_oe property

Track if we are in a transaction or not - allows saving multiple models to a document
protected bool $_transactionStarted
Résultat boolean

$_type protected_oe property

If we are in a transaction this is the 'type' for the model indexing
protected string $_type
Résultat string

$columns public_oe property

Elasticsearch column definition
public array $columns
Résultat array

$fullDebug public_oe property

Prevents error from being thrown in CakeTestFixture
public string $fullDebug
Résultat string

$useNestedTransactions public_oe property

Prevents error from being thrown in CakeTestFixture
public string $useNestedTransactions
Résultat string