tag:blogger.com,1999:blog-826910957631479390.post3137230634571948991..comments2024-03-28T03:15:27.747-07:00Comments on Evol Monkey: PostgreSQL and ElasticSearchEvol Monkeyhttp://www.blogger.com/profile/01346397854558520528noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-826910957631479390.post-69016972362010457102014-08-26T13:57:18.472-07:002014-08-26T13:57:18.472-07:00yeah, i was thinking something similar in case i u...yeah, i was thinking something similar in case i used this for production. Something else that would work would be a status column stating the "replication status" of a row, that way would give you a potential flexibility if you want to partially replicate "some" data.Evol Monkeyhttps://www.blogger.com/profile/01346397854558520528noreply@blogger.comtag:blogger.com,1999:blog-826910957631479390.post-8044541309766511222014-08-26T13:26:39.258-07:002014-08-26T13:26:39.258-07:00I take a similar approach and it has been work wel...I take a similar approach and it has been work well. One thing to be wary of when relying on the content of the notification itself for indexing is that you risk missing some rows if they are inserted whilst your python script is not running.<br /><br />To avoid this, I have an xid column in my table with a default of txid_current(). My ES indexing script (triggered by the notification in a similar select loop) then finds the last_indexed_xmin from an ancillary document in ES and indexes all rows with xid >= last_indexed_xmin, finally updating the last_indexed_xmin to the current txid_snapshot_xmin(txid_current_snapshot()) in the ancillary document.Laurence Rowehttps://www.blogger.com/profile/02164563755538372331noreply@blogger.com