Thursday, March 5, 2020

How to fix the concurrent update issue in ElasticSearch with Node.js development?

if we try to update the information from two different source with ElasticSearch.


we may encounter a concurrent issue. ElasticSearch has mechanish to prevent the concurrent update.

here is the sample code to get information from bundle and book information for response.


app.put('/api/bundle/:id/book/:pgid'async(reqres)=>{
        const bundleUrl =`${url}/${req.params.id}`;
        const bookUrl=`http://${es.host}:${es.port}/${es.books_index}/book/${req.params.pgid}`;
        try{
            const [bundleResbookRes]=await Promise.all([
                rp({url:bundleUrljson:true}),
                rp({url:bookUrljson:true})
            ]);
            console.log(bundleRes);
            const {_sourcebundle_versionversion}=bundleRes;
            const {_sourcebook}=bookRes;
            console.log("if_seq_no:",if_seq_no);
            console.log("if_primary_term"if_primary_term);
            const idx=bundle.books.findIndex(book=>book.id===req.params.pgid);
            if(idx === -1){
                bundle.books.push({
                    id: req.params.pgid,
                    title: book.title
                });
            }
            const esResBody=await rp.put({
                url:bundleUrl,
                qs:{
  version: version
                    version_type: 'external',
                },
                body:bundle,
                json:true
            });
            res.status(200).json(esResBody);

        }catch(esResErr){
            res.status(esResErr.status ||502).json(esResErr.error);
        }

    })


initially i try to use the version number to resolve the concurrent issue. the error is thrown immediately after the request is made.

{
    "error": {
        "root_cause": [
            {
                "type": "action_request_validation_exception",
                "reason": "Validation Failed: 1: internal versioning can not be used for optimistic concurrency control. Please use `if_seq_no` and `if_primary_term` instead;"
            }
        ],
        "type": "action_request_validation_exception",
        "reason": "Validation Failed: 1: internal versioning can not be used for optimistic concurrency control. Please use `if_seq_no` and `if_primary_term` instead;"
    },
    "status": 400
}



based on the suggestion from the error message. i need to make couple of change, in order to fix the issue.

first we need to load the _seq_no and _primary_term from bundle response body.

 const {_sourcebundle_versionversion,  
 _seq_noif_seq_no
 _primary_term
 if_primary_term }=bundleRes;
 const {_sourcebook}=bookRes;

second, add if_seq_no and if_primary_term to the query string array 
const esResBody=await rp.put({
                url:bundleUrl,
                qs:{
                    if_seq_no: if_seq_no,
                    if_primary_term: if_primary_term
                },
                body:bundle,
                json:true
            });


Now the concurrent issue is gone. the title update is done,

 danhuideng@DESKTOP-EE785O3:/mnt/c/node/b4$ curl -s localhost:60702/api/bundle/$BUNDLE_ID |jq '._source'
{
  "name": "far",
  "books": [
    {
      "id": "pg132",
      "title": "The Art of War"
    }
  ]
}



No comments:

Post a Comment