Added support for supplying hints to various SODA operations.

This commit is contained in:
Anthony Tuininga 2021-04-15 16:59:34 -06:00
parent 74b2e9a258
commit 517868eaf9
5 changed files with 179 additions and 23 deletions

View File

@ -247,19 +247,29 @@ SODA Collection Object
.. versionadded:: 7.2 .. versionadded:: 7.2
.. method:: SodaCollection.insertManyAndGet(docs) .. method:: SodaCollection.insertManyAndGet(docs, hint=None)
Similarly to :meth:`~SodaCollection.insertMany()` this method inserts a Similarly to :meth:`~SodaCollection.insertMany()` this method inserts a
list of documents into the collection at one time. The only difference is list of documents into the collection at one time. The only difference is
that it returns a list of :ref:`SODA Document objects <sodadoc>`. Note that that it returns a list of :ref:`SODA Document objects <sodadoc>`. Note that
for performance reasons the returned documents do not contain the content. for performance reasons the returned documents do not contain the content.
The hint parameter, if specified, supplies a hint to the database when
processing the SODA operation. This is expected to be a string in the same
format as SQL hints but without the enclosing comment characters. Use of
this parameter requires Oracle Client 21.3 or higher (or Oracle Client 19
from 19.11).
.. note:: .. note::
This method requires Oracle Client 18.5 and higher. This method requires Oracle Client 18.5 and higher.
.. versionadded:: 7.2 .. versionadded:: 7.2
.. versionchanged:: 8.2
The parameter `hint` was added.
.. method:: SodaCollection.insertOne(doc) .. method:: SodaCollection.insertOne(doc)
@ -269,15 +279,25 @@ SODA Collection Object
.. versionadded:: 7.0 .. versionadded:: 7.0
.. method:: SodaCollection.insertOneAndGet(doc) .. method:: SodaCollection.insertOneAndGet(doc, hint=None)
Similarly to :meth:`~SodaCollection.insertOne()` this method inserts a Similarly to :meth:`~SodaCollection.insertOne()` this method inserts a
given document into the collection. The only difference is that it given document into the collection. The only difference is that it
returns a :ref:`SODA Document object <sodadoc>`. Note that for performance returns a :ref:`SODA Document object <sodadoc>`. Note that for performance
reasons the returned document does not contain the content. reasons the returned document does not contain the content.
The hint parameter, if specified, supplies a hint to the database when
processing the SODA operation. This is expected to be a string in the same
format as SQL hints but without the enclosing comment characters. Use of
this parameter requires Oracle Client 21.3 or higher (or Oracle Client 19
from 19.11).
.. versionadded:: 7.0 .. versionadded:: 7.0
.. versionchanged:: 8.2
The parameter `hint` was added.
.. attribute:: SodaCollection.metadata .. attribute:: SodaCollection.metadata
@ -310,18 +330,28 @@ SODA Collection Object
.. versionadded:: 8.0 .. versionadded:: 8.0
.. method:: SodaCollection.saveAndGet(doc) .. method:: SodaCollection.saveAndGet(doc, hint=None)
Saves a document into the collection. This method is equivalent to Saves a document into the collection. This method is equivalent to
:meth:`~SodaCollection.insertOneAndGet()` except that if client-assigned :meth:`~SodaCollection.insertOneAndGet()` except that if client-assigned
keys are used, and the document with the specified key already exists in keys are used, and the document with the specified key already exists in
the collection, it will be replaced with the input document. the collection, it will be replaced with the input document.
The hint parameter, if specified, supplies a hint to the database when
processing the SODA operation. This is expected to be a string in the same
format as SQL hints but without the enclosing comment characters. Use of
this parameter requires Oracle Client 21.3 or higher (or Oracle Client 19
from 19.11).
This method requires Oracle Client 19.9 or higher in addition to the usual This method requires Oracle Client 19.9 or higher in addition to the usual
SODA requirements. SODA requirements.
.. versionadded:: 8.0 .. versionadded:: 8.0
.. versionchanged:: 8.2
The parameter `hint` was added.
.. method:: SodaCollection.truncate() .. method:: SodaCollection.truncate()
@ -526,6 +556,19 @@ SODA Operation Object
.. versionadded:: 7.0 .. versionadded:: 7.0
.. method:: SodaOperation.hint(value)
Specifies a hint that will be provided to the SODA operation when it is
performed. This is expected to be a string in the same format as SQL hints
but without the enclosing comment characters. Use of this method
requires Oracle Client 21.3 or higher (or Oracle Client 19 from 19.11).
As a convenience, the SodaOperation object is returned so that further
criteria can be specified by chaining methods together.
.. versionadded:: 8.2
.. method:: SodaOperation.key(value) .. method:: SodaOperation.key(value)
Specifies that the document with the specified key should be returned. Specifies that the document with the specified key should be returned.

View File

@ -11,6 +11,12 @@ Version 8.2 (TBD)
#) Updated embedded ODPI-C to `version 4.2.0 #) Updated embedded ODPI-C to `version 4.2.0
<https://oracle.github.io/odpi/doc/releasenotes.html# <https://oracle.github.io/odpi/doc/releasenotes.html#
version-4-2-tbd>`__. version-4-2-tbd>`__.
#) Added support for supplying hints to SODA operations. A new non-terminal
method :meth:`~SodaOperation.hint()` was added and a `hint` parameter was
added to the methods :meth:`SodaCollection.insertOneAndGet()`,
:meth:`SodaCollection.insertManyAndGet()` and
:meth:`SodaCollection.saveAndGet()`. All of these require Oracle Client
21.3 or higher (or Oracle Client 19 from 19.11).
#) Eliminated memory leak when calling :meth:`SodaOperation.filter()` with a #) Eliminated memory leak when calling :meth:`SodaOperation.filter()` with a
dictionary. dictionary.
#) The distributed transaction handle assosciated with the connection is now #) The distributed transaction handle assosciated with the connection is now

View File

@ -436,6 +436,7 @@ struct cxoSodaOperation {
cxoBuffer keyBuffer; cxoBuffer keyBuffer;
cxoBuffer versionBuffer; cxoBuffer versionBuffer;
cxoBuffer filterBuffer; cxoBuffer filterBuffer;
cxoBuffer hintBuffer;
}; };

View File

@ -9,6 +9,26 @@
#include "cxoModule.h" #include "cxoModule.h"
//-----------------------------------------------------------------------------
// cxoSodaCollection_processOptions()
// Populate the SODA operations structure with the information provided by
// the user.
//-----------------------------------------------------------------------------
static int cxoSodaCollection_processOptions(cxoSodaCollection *coll,
dpiSodaOperOptions *options, PyObject *hintObj, cxoBuffer *hintBuffer)
{
if (dpiContext_initSodaOperOptions(cxoDpiContext, options) < 0)
return cxoError_raiseAndReturnInt();
if (cxoBuffer_fromObject(hintBuffer, hintObj,
coll->db->connection->encodingInfo.encoding) < 0)
return -1;
options->hint = hintBuffer->ptr;
options->hintLength = hintBuffer->size;
return 0;
}
//----------------------------------------------------------------------------- //-----------------------------------------------------------------------------
// cxoSodaCollection_initialize() // cxoSodaCollection_initialize()
// Initialize a new collection with its attributes. // Initialize a new collection with its attributes.
@ -228,7 +248,7 @@ static PyObject *cxoSodaCollection_getDataGuide(cxoSodaCollection *coll,
//----------------------------------------------------------------------------- //-----------------------------------------------------------------------------
static PyObject *cxoSodaCollection_insertManyHelper(cxoSodaCollection *coll, static PyObject *cxoSodaCollection_insertManyHelper(cxoSodaCollection *coll,
PyObject *docs, Py_ssize_t numDocs, dpiSodaDoc **handles, PyObject *docs, Py_ssize_t numDocs, dpiSodaDoc **handles,
dpiSodaDoc **returnHandles) dpiSodaDoc **returnHandles, dpiSodaOperOptions *options)
{ {
PyObject *element, *returnDocs; PyObject *element, *returnDocs;
Py_ssize_t i, j; Py_ssize_t i, j;
@ -252,8 +272,8 @@ static PyObject *cxoSodaCollection_insertManyHelper(cxoSodaCollection *coll,
// perform bulk insert // perform bulk insert
Py_BEGIN_ALLOW_THREADS Py_BEGIN_ALLOW_THREADS
status = dpiSodaColl_insertMany(coll->handle, (uint32_t) numDocs, handles, status = dpiSodaColl_insertManyWithOptions(coll->handle,
flags, returnHandles); (uint32_t) numDocs, handles, options, flags, returnHandles);
Py_END_ALLOW_THREADS Py_END_ALLOW_THREADS
if (status < 0) if (status < 0)
cxoError_raiseAndReturnNull(); cxoError_raiseAndReturnNull();
@ -309,7 +329,7 @@ static PyObject *cxoSodaCollection_insertMany(cxoSodaCollection *coll,
return NULL; return NULL;
} }
result = cxoSodaCollection_insertManyHelper(coll, arg, numDocs, handles, result = cxoSodaCollection_insertManyHelper(coll, arg, numDocs, handles,
NULL); NULL, NULL);
PyMem_Free(handles); PyMem_Free(handles);
return result; return result;
} }
@ -321,32 +341,52 @@ static PyObject *cxoSodaCollection_insertMany(cxoSodaCollection *coll,
// list of documents containing all but the content itself. // list of documents containing all but the content itself.
//----------------------------------------------------------------------------- //-----------------------------------------------------------------------------
static PyObject *cxoSodaCollection_insertManyAndGet(cxoSodaCollection *coll, static PyObject *cxoSodaCollection_insertManyAndGet(cxoSodaCollection *coll,
PyObject *arg) PyObject *args, PyObject *keywordArgs)
{ {
static char *keywordList[] = { "docs", "hint", NULL };
dpiSodaOperOptions options, *optionsPtr = NULL;
dpiSodaDoc **handles, **returnHandles; dpiSodaDoc **handles, **returnHandles;
PyObject *docsObj, *hintObj, *result;
cxoBuffer hintBuffer;
Py_ssize_t numDocs; Py_ssize_t numDocs;
PyObject *result;
if (!PyList_Check(arg)) { // parse arguments
docsObj = hintObj = NULL;
if (!PyArg_ParseTupleAndKeywords(args, keywordArgs, "O|O", keywordList,
&docsObj, &hintObj))
return NULL;
if (!PyList_Check(docsObj)) {
PyErr_SetString(PyExc_TypeError, "expecting list"); PyErr_SetString(PyExc_TypeError, "expecting list");
return NULL; return NULL;
} }
numDocs = PyList_GET_SIZE(arg);
// setup for actual work
cxoBuffer_init(&hintBuffer);
if (hintObj && hintObj != Py_None) {
optionsPtr = &options;
if (cxoSodaCollection_processOptions(coll, &options, hintObj,
&hintBuffer) < 0)
return NULL;
}
numDocs = PyList_GET_SIZE(docsObj);
handles = PyMem_Malloc(numDocs * sizeof(dpiSodaDoc*)); handles = PyMem_Malloc(numDocs * sizeof(dpiSodaDoc*));
if (!handles) { if (!handles) {
PyErr_NoMemory(); PyErr_NoMemory();
cxoBuffer_clear(&hintBuffer);
return NULL; return NULL;
} }
returnHandles = PyMem_Malloc(numDocs * sizeof(dpiSodaDoc*)); returnHandles = PyMem_Malloc(numDocs * sizeof(dpiSodaDoc*));
if (!returnHandles) { if (!returnHandles) {
PyErr_NoMemory(); PyErr_NoMemory();
PyMem_Free(handles); PyMem_Free(handles);
cxoBuffer_clear(&hintBuffer);
return NULL; return NULL;
} }
result = cxoSodaCollection_insertManyHelper(coll, arg, numDocs, handles, result = cxoSodaCollection_insertManyHelper(coll, docsObj, numDocs,
returnHandles); handles, returnHandles, optionsPtr);
PyMem_Free(handles); PyMem_Free(handles);
PyMem_Free(returnHandles); PyMem_Free(returnHandles);
cxoBuffer_clear(&hintBuffer);
return result; return result;
} }
@ -384,23 +424,46 @@ static PyObject *cxoSodaCollection_insertOne(cxoSodaCollection *coll,
// containing all but the content itself. // containing all but the content itself.
//----------------------------------------------------------------------------- //-----------------------------------------------------------------------------
static PyObject *cxoSodaCollection_insertOneAndGet(cxoSodaCollection *coll, static PyObject *cxoSodaCollection_insertOneAndGet(cxoSodaCollection *coll,
PyObject *arg) PyObject *args, PyObject *keywordArgs)
{ {
static char *keywordList[] = { "doc", "hint", NULL };
dpiSodaOperOptions options, *optionsPtr = NULL;
dpiSodaDoc *handle, *returnedHandle; dpiSodaDoc *handle, *returnedHandle;
PyObject *docObj, *hintObj;
cxoBuffer hintBuffer;
uint32_t flags; uint32_t flags;
int status; int status;
if (cxoUtils_processSodaDocArg(coll->db, arg, &handle) < 0) // parse arguments
docObj = hintObj = NULL;
if (!PyArg_ParseTupleAndKeywords(args, keywordArgs, "O|O", keywordList,
&docObj, &hintObj))
return NULL; return NULL;
// setup for actual work
if (cxoConnection_getSodaFlags(coll->db->connection, &flags) < 0) if (cxoConnection_getSodaFlags(coll->db->connection, &flags) < 0)
return NULL; return NULL;
if (cxoUtils_processSodaDocArg(coll->db, docObj, &handle) < 0)
return NULL;
cxoBuffer_init(&hintBuffer);
if (hintObj && hintObj != Py_None) {
optionsPtr = &options;
if (cxoSodaCollection_processOptions(coll, &options, hintObj,
&hintBuffer) < 0) {
dpiSodaDoc_release(handle);
return NULL;
}
}
// perform actual work
Py_BEGIN_ALLOW_THREADS Py_BEGIN_ALLOW_THREADS
status = dpiSodaColl_insertOne(coll->handle, handle, flags, status = dpiSodaColl_insertOneWithOptions(coll->handle, handle, optionsPtr,
&returnedHandle); flags, &returnedHandle);
Py_END_ALLOW_THREADS Py_END_ALLOW_THREADS
if (status < 0) if (status < 0)
cxoError_raiseAndReturnNull(); cxoError_raiseAndReturnNull();
dpiSodaDoc_release(handle); dpiSodaDoc_release(handle);
cxoBuffer_clear(&hintBuffer);
if (status < 0) if (status < 0)
return NULL; return NULL;
return (PyObject*) cxoSodaDoc_new(coll->db, returnedHandle); return (PyObject*) cxoSodaDoc_new(coll->db, returnedHandle);
@ -463,22 +526,46 @@ static PyObject *cxoSodaCollection_save(cxoSodaCollection *coll,
// containing all but the content itself. // containing all but the content itself.
//----------------------------------------------------------------------------- //-----------------------------------------------------------------------------
static PyObject *cxoSodaCollection_saveAndGet(cxoSodaCollection *coll, static PyObject *cxoSodaCollection_saveAndGet(cxoSodaCollection *coll,
PyObject *arg) PyObject *args, PyObject *keywordArgs)
{ {
static char *keywordList[] = { "doc", "hint", NULL };
dpiSodaOperOptions options, *optionsPtr = NULL;
dpiSodaDoc *handle, *returnedHandle; dpiSodaDoc *handle, *returnedHandle;
PyObject *docObj, *hintObj;
cxoBuffer hintBuffer;
uint32_t flags; uint32_t flags;
int status; int status;
if (cxoUtils_processSodaDocArg(coll->db, arg, &handle) < 0) // parse arguments
docObj = hintObj = NULL;
if (!PyArg_ParseTupleAndKeywords(args, keywordArgs, "O|O", keywordList,
&docObj, &hintObj))
return NULL; return NULL;
// setup for actual work
if (cxoConnection_getSodaFlags(coll->db->connection, &flags) < 0) if (cxoConnection_getSodaFlags(coll->db->connection, &flags) < 0)
return NULL; return NULL;
if (cxoUtils_processSodaDocArg(coll->db, docObj, &handle) < 0)
return NULL;
cxoBuffer_init(&hintBuffer);
if (hintObj && hintObj != Py_None) {
optionsPtr = &options;
if (cxoSodaCollection_processOptions(coll, &options, hintObj,
&hintBuffer) < 0) {
dpiSodaDoc_release(handle);
return NULL;
}
}
// perform actual work
Py_BEGIN_ALLOW_THREADS Py_BEGIN_ALLOW_THREADS
status = dpiSodaColl_save(coll->handle, handle, flags, &returnedHandle); status = dpiSodaColl_saveWithOptions(coll->handle, handle, optionsPtr,
flags, &returnedHandle);
Py_END_ALLOW_THREADS Py_END_ALLOW_THREADS
if (status < 0) if (status < 0)
cxoError_raiseAndReturnNull(); cxoError_raiseAndReturnNull();
dpiSodaDoc_release(handle); dpiSodaDoc_release(handle);
cxoBuffer_clear(&hintBuffer);
if (status < 0) if (status < 0)
return NULL; return NULL;
return (PyObject*) cxoSodaDoc_new(coll->db, returnedHandle); return (PyObject*) cxoSodaDoc_new(coll->db, returnedHandle);
@ -516,12 +603,13 @@ static PyMethodDef cxoMethods[] = {
METH_NOARGS }, METH_NOARGS },
{ "insertOne", (PyCFunction) cxoSodaCollection_insertOne, METH_O }, { "insertOne", (PyCFunction) cxoSodaCollection_insertOne, METH_O },
{ "insertOneAndGet", (PyCFunction) cxoSodaCollection_insertOneAndGet, { "insertOneAndGet", (PyCFunction) cxoSodaCollection_insertOneAndGet,
METH_O }, METH_VARARGS | METH_KEYWORDS },
{ "insertMany", (PyCFunction) cxoSodaCollection_insertMany, METH_O }, { "insertMany", (PyCFunction) cxoSodaCollection_insertMany, METH_O },
{ "insertManyAndGet", (PyCFunction) cxoSodaCollection_insertManyAndGet, { "insertManyAndGet", (PyCFunction) cxoSodaCollection_insertManyAndGet,
METH_O }, METH_VARARGS | METH_KEYWORDS },
{ "save", (PyCFunction) cxoSodaCollection_save, METH_O }, { "save", (PyCFunction) cxoSodaCollection_save, METH_O },
{ "saveAndGet", (PyCFunction) cxoSodaCollection_saveAndGet, METH_O }, { "saveAndGet", (PyCFunction) cxoSodaCollection_saveAndGet,
METH_VARARGS | METH_KEYWORDS },
{ "truncate", (PyCFunction) cxoSodaCollection_truncate, METH_NOARGS }, { "truncate", (PyCFunction) cxoSodaCollection_truncate, METH_NOARGS },
{ NULL } { NULL }
}; };

View File

@ -130,6 +130,23 @@ static PyObject *cxoSodaOperation_filter(cxoSodaOperation *op,
} }
//-----------------------------------------------------------------------------
// cxoSodaOperation_hint()
// Set the hint to be used for the operation.
//-----------------------------------------------------------------------------
static PyObject *cxoSodaOperation_hint(cxoSodaOperation *op, PyObject *hintObj)
{
cxoBuffer_clear(&op->hintBuffer);
if (cxoBuffer_fromObject(&op->hintBuffer, hintObj,
op->coll->db->connection->encodingInfo.encoding) < 0)
return NULL;
op->options.hint = op->hintBuffer.ptr;
op->options.hintLength = op->hintBuffer.size;
Py_INCREF(op);
return (PyObject*) op;
}
//----------------------------------------------------------------------------- //-----------------------------------------------------------------------------
// cxoSodaOperation_key() // cxoSodaOperation_key()
// Set the key to be used for the operation. // Set the key to be used for the operation.
@ -502,6 +519,7 @@ static PyMethodDef cxoMethods[] = {
{ "getDocuments", (PyCFunction) cxoSodaOperation_getDocuments, { "getDocuments", (PyCFunction) cxoSodaOperation_getDocuments,
METH_NOARGS }, METH_NOARGS },
{ "getOne", (PyCFunction) cxoSodaOperation_getOne, METH_NOARGS }, { "getOne", (PyCFunction) cxoSodaOperation_getOne, METH_NOARGS },
{ "hint", (PyCFunction) cxoSodaOperation_hint, METH_O },
{ "remove", (PyCFunction) cxoSodaOperation_remove, METH_NOARGS }, { "remove", (PyCFunction) cxoSodaOperation_remove, METH_NOARGS },
{ "replaceOne", (PyCFunction) cxoSodaOperation_replaceOne, METH_O }, { "replaceOne", (PyCFunction) cxoSodaOperation_replaceOne, METH_O },
{ "replaceOneAndGet", (PyCFunction) cxoSodaOperation_replaceOneAndGet, { "replaceOneAndGet", (PyCFunction) cxoSodaOperation_replaceOneAndGet,