Key Idea

• Instead of insertEvents.js calling AddEvent directly, it will call a simple insert function to enqueue into a class Queue (similar to test_ProcessCreate)
• Capture the events quickly first, scale out can come later

Main Goal

For this internship, measure the difference between throughput compared to https://www.notion.so/jymcheong/AddEvent-Throughput-3b2e5ee9b96e4c9db206d552908be3c3, meaning AddEvent vs Enqueue Function

Clarifications

Inserting "escaped" JSON string. Line 6 of test_ProcessCreate has a escape(JSON.stringify(pc)). This function returns a string, we want to insert that string into a Queue class.

This escaping of string is important, otherwise there will be problems.

Your last statement was:

so from this u can see the amount of time that can be saved if the events have already been processed before insertion am i right?

When we enqueue escaped string of an incoming event, it is NOT processed at all yet. I will be re-writing several ODB functions to Dequeue & change the way we derive Sequence. But AddEvent will largely be similar.

So this task is not so much of "time saved", but rather "timing difference" between a complex function like AddEvent vs a simple INSERT INTO QUEUE EventString = ... where ... is the escaped JSON object.

YJ's Documentation

Aim

Measure and compare the time it takes to insert 10000 ProcessCreate events using a simple enqueue function vs AddEvent function.

Implementation Steps

1. Create a new class called Queue in ODB. Create a property called EventString in class Queue to store the escaped JSON object.

   

2. Modify test_ProcessCreate function to insert the event into class Queue instead of calling AddEvent function.

   db.command('INSERT INTO Queue SET EventString = ?',  escape(JSON.stringify(pc)))
   

3. Use the test function that was used in https://www.notion.so/jymcheong/AddEvent-Throughput-3b2e5ee9b96e4c9db206d552908be3c3 to measure the insertion performance.

   //shortcut to get ticks in milliseconds since epoch
   var s = (new Date())*1
   
   print('start: ' + s)
   print(Date())
   
   for(var i=0;i < 10000;i++)
     test_ProcessCreate()
   
   var e = (new Date())*1
   
   print('end: ' + e)
   print(Date())
   
   var d = (e - s)/1000
   
   print('====')
   print('delta in secs ' + d)
   print('====')