Skip to content

Home

Jump to bottom
Rose Heart edited this page Feb 20, 2025 · 14 revisions

JackrabbitDLM Wiki

Introduction

JackrabbitDLM is a Distributed Lock Manager (DLM) designed to manage locks and temporary data storage in a networked environment. It operates as a daemon, listening on a specified port for incoming requests in JSON format. These requests can include actions such as locking, unlocking, retrieving, storing, and erasing memory references. The system ensures that lock ownership is maintained by associating each lock with a unique ID, preventing unauthorized access. If a lock expires or is explicitly released, it becomes available for reassignment. The program logs all actions for auditing and debugging purposes. By leveraging a non-blocking socket interface, JackrabbitDLM efficiently handles multiple simultaneous connections, making it suitable for distributed applications requiring concurrency control and temporary state management.

Security and Access Control

Access control is the responsibility of the user's firewall. JackrabbitDLM does not provide authentication or access restrictions, and Python is NOT a substitution for proper system security. Users must ensure that their firewall and network configurations restrict access to authorized clients only. It is crucial to implement proper security measures to prevent unauthorized access to the service.

Features

  • Distributed Lock Management: Ensures resource locking across multiple clients.
  • Temporary Data Storage: Supports storing and retrieving data with expiration control.
  • Concurrency Control: Handles multiple simultaneous requests using non-blocking sockets.
  • Security and Ownership Control: Prevents unauthorized access to locked resources.

How It Works

  1. Listening for Requests: The daemon listens on a specified port for incoming JSON-formatted requests.
  2. Processing Payloads: Each request contains an action (Lock, Unlock, Get, Put, Erase) and relevant parameters such as ID, FileName, and Expiry.
  3. Lock Management:
    • New locks are assigned to the requesting ID.
    • Expired locks are reset and reassigned.
    • Unauthorized unlock or modification attempts are denied.
  4. Data Storage:
    • Put stores data with an expiration time.
    • Get retrieves stored data if the requesting ID owns it.
    • Erase removes stored data upon owner request.
  5. Non-blocking Operation: The system handles multiple connections simultaneously without blocking, improving efficiency in distributed environments.

JSON Request Examples

  • Lock a Resource: 
    { "ID":"DEADBEEF", "FileName":"testData", "Action":"Lock", "Expire":"300" }
  • Unlock a Resource: 
    { "ID":"DEADBEEF", "FileName":"testData", "Action":"Unlock" }
  • Store Data: 
    { "ID":"DEADBEEF", "FileName":"testData", "Action":"Put", "Expire":"300", "DataStore":"Blah" }
  • Retrieve Data: 
    { "ID":"DEADBEEF", "FileName":"testData", "Action":"Get" }
  • Erase Data: 
    { "ID":"DEADBEEF", "FileName":"testData", "Action":"Erase" }

Usage

To start the JackrabbitDLM service, run the script with an optional host and port:

./JackrabbitDLM [host] [port]

By default, it binds to all interfaces (0.0.0.0) and listens on port 37373.

Supporting Library

The Locker class in JackrabbitDLM provides a robust framework for managing distributed file locks over a network. It allows clients to acquire, check, and release locks on shared resources using a JSON-based protocol. The class generates unique IDs for each client, ensuring secure lock ownership. The Talker method facilitates communication with the locking server, while the Retry and RetryData functions handle request retries for reliability. The class supports fundamental operations such as Lock and Unlock, along with data management functions like Put, Get, and Erase, allowing users to store and retrieve data associated with locked resources. The implementation prioritizes efficiency through retry logic and configurable timeout settings, ensuring smooth operation in networked environments.

Testing and Example

JackrabbitDLM can be stress-tested using a simulation known as a Lock War, which demonstrates the library’s ability to handle extreme contention scenarios. This test aggressively competes for access to a shared resource, simulating worst-case conditions where multiple processes, referred to as fighters, initiate simultaneous lock, read, and write operations. The number of fighters is specified at runtime, allowing for controlled scalability of the test. Each fighter operates in either Retry Mode (internal to the Locker class), where lock attempts are retried until successful, or Hyper-Aggressive Mode, where a single lock attempt is made without retries.

The simulation launches multiple Locker instances—one per fighter—where each process attempts to acquire a lock on a shared resource. If successful, the process retrieves stored data from memory, increments a counter if data exists, and writes the updated value back 25% of the time. If no data is found, it initializes the memory store. This repeated cycle of lock acquisition, reading, and writing places the DLM under significant load, simulating real-world contention where multiple clients compete for limited resources.

A shell script orchestrates the execution of multiple fighters, each running independently to maximize stress on the DLM. The number of fighters and the mode of operation are passed as parameters, allowing for dynamic test configurations. Performance metrics are collected, including the number of successful and failed lock attempts, read and write operations, and the contention rate—the percentage of failed attempts over total lock requests. The program logs these results along with execution time and process ID, providing valuable insights into the DLM’s efficiency and stability under high-demand conditions. This simulation serves as both a usage demonstration and a testbed for evaluating JackrabbitDLM’s ability to handle concurrent operations at scale.

Operational Statistics and logging

Here is an example of hourly statistics logged by Jackrabbit DLM and the meaning of each item:

  • AIn, This captures the count of active incoming requests currently being processed, providing a real-time snapshot of system activity.
  • ALock, This reflects the number of active locks currently held within JackrabbitDLM, indicating resource utilization.
  • AOut, This counts active outbound requests that have been initiated but not yet completed, helping assess pending responses.
  • Erased, Tracks data items erased using the .Erase() function, highlighting data management practices.
  • ExpiredData. Indicates how many data items were removed due to expiration policies during that hour.
  • Expired, This tracks the number of locks that were freed due to expiration, indicating how many resources became available again after their designated time limit was reached.
  • Get, Counts all requests made to retrieve stored data within an hour.
  • GetNF, Reflects instances where Get requests were unsuccessful due to missing or uninitialized data.
  • In, Captures total inbound requests processed over an hour, giving an overview of overall system activity.
  • Lock, Tracks all lock requests processed in the last hour, indicating client attempts to secure resources through locking mechanisms.
  • NotOwner, Shows failed attempts where access was denied because another process owned the lock.
  • Out, Captures total outbound responses during that hour.
  • PutNew, Counts new data items added during that timeframe.
  • PutUpdate, Tracks updates made to existing data items stored in JackrabbitDLM over an hour.
  • Unlock, Captures all unlock requests processed during that hour.
  • UnlockNF, This metric captures requests to unlock a resource, but the resource is not currently being tracked.
2025-02-19 18:13:14.843674 Jackrabbit DLM 0.0.0.0.125
2025-02-19 18:13:15.377045 AIn: 1, In: 1
2025-02-19 19:00:00.005473 AData: 3, AIn: 1, ALock: 2, AOut: 1, ExpiredData: 177257, Get: 3356, GetNF: 7451, In: 1415544, Lock: 112582, NotOwner: 26441, Out: 951660, PutNew: 177260, PutUpdate: 15706, Unlock: 129052
2025-02-19 20:00:00.000622 AData: 1, AIn: 1, ALock: 3, AOut: 1, ExpiredData: 227614, Get: 4069, GetNF: 9606, In: 1814901, Lock: 144513, NotOwner: 33928, Out: 1216444, PutNew: 227612, PutUpdate: 20001, Unlock: 165238
2025-02-19 21:00:00.007723 AData: 2, AIn: 1, ALock: 4, AOut: 1, ExpiredData: 227645, Get: 4210, GetNF: 9572, In: 1816164, Lock: 144134, NotOwner: 34793, Out: 1217352, PutNew: 227646, PutUpdate: 20035, Unlock: 164998
2025-02-19 22:00:00.005556 AData: 3, AIn: 1, ALock: 4, AOut: 1, ExpiredData: 226971, Get: 4661, GetNF: 9395, In: 1793838, Lock: 141779, NotOwner: 33407, Out: 1202452, PutNew: 226972, PutUpdate: 19582, Unlock: 162150
2025-02-19 23:00:00.006504 AData: 5, AIn: 1, ALock: 3, AOut: 1, ExpiredData: 226609, Get: 4698, GetNF: 9373, In: 1790157, Lock: 141624, NotOwner: 32745, Out: 1199948, PutNew: 226611, PutUpdate: 19602, Unlock: 162066
2025-02-20 00:00:00.013259 AData: 3, AIn: 1, ALock: 2, AOut: 1, ExpiredData: 228630, Get: 4635, GetNF: 9405, In: 1808331, Lock: 142188, NotOwner: 35525, Out: 1212417, PutNew: 228628, PutUpdate: 19683, Unlock: 162713
2025-02-20 01:00:00.001020 AData: 5, AIn: 1, ALock: 2, AOut: 1, Expired: 1, ExpiredData: 221693, Get: 4482, GetNF: 9166, In: 1755004, Lock: 138674, NotOwner: 33659, Out: 1193232, PutNew: 221695, PutUpdate: 18985, Unlock: 158340
2025-02-20 02:00:00.009703 AData: 1, AIn: 1, ALock: 3, AOut: 1, ExpiredData: 228605, Get: 4558, GetNF: 9469, In: 1811942, Lock: 143018, NotOwner: 35071, Out: 1214290, PutNew: 228601, PutUpdate: 19744, Unlock: 163519
2025-02-20 03:00:00.000361 AData: 1, AIn: 1, ALock: 3, AOut: 1, ExpiredData: 228017, Get: 4124, GetNF: 9649, In: 1826142, Lock: 145191, NotOwner: 35542, Out: 1224477, PutNew: 228017, PutUpdate: 20117, Unlock: 166074
2025-02-20 04:00:00.008355 AData: 5, AIn: 1, ALock: 2, AOut: 1, ExpiredData: 228289, Get: 4071, GetNF: 9684, In: 1834488, Lock: 145704, NotOwner: 36643, Out: 1230020, PutNew: 228293, PutUpdate: 20280, Unlock: 166821
2025-02-20 05:00:00.003663 AData: 2, AIn: 1, ALock: 1, AOut: 1, ExpiredData: 229322, Get: 4198, GetNF: 9729, In: 1846296, Lock: 146329, NotOwner: 38026, Out: 1237957, PutNew: 229319, PutUpdate: 20346, Unlock: 167485
2025-02-20 06:00:00.000182 AData: 2, AIn: 2, ALock: 2, AOut: 1, ExpiredData: 230508, Get: 4262, GetNF: 9792, In: 1864893, Lock: 147479, NotOwner: 40199, Out: 1250580, PutNew: 230508, PutUpdate: 20535, Unlock: 168856
2025-02-20 07:00:00.000864 AData: 2, AIn: 1, ALock: 3, AOut: 1, ExpiredData: 231394, Get: 4288, GetNF: 9839, In: 1869909, Lock: 147780, NotOwner: 40557, Out: 1253762, PutNew: 231394, PutUpdate: 20479, Unlock: 168966
2025-02-20 08:00:00.000478 AData: 4, AIn: 2, ALock: 2, AOut: 1, ExpiredData: 231493, Get: 4371, GetNF: 9829, In: 1870059, Lock: 147593, NotOwner: 40836, Out: 1253882, PutNew: 231495, PutUpdate: 20450, Unlock: 168779
2025-02-20 09:00:00.005268 AData: 4, AIn: 1, ALock: 4, AOut: 1, ExpiredData: 232102, Get: 4349, GetNF: 9865, In: 1878864, Lock: 148419, NotOwner: 41181, Out: 1259796, PutNew: 232102, PutUpdate: 20577, Unlock: 169795
2025-02-20 10:00:00.005994 AData: 2, AIn: 1, ALock: 3, AOut: 1, Expired: 1, ExpiredData: 232037, Get: 4336, GetNF: 9877, In: 1877640, Lock: 148275, NotOwner: 41254, Out: 1259119, PutNew: 232035, PutUpdate: 20546, Unlock: 169557
2025-02-20 11:00:00.000360 AData: 5, AIn: 1, ALock: 4, AOut: 1, Expired: 1, ExpiredData: 231858, Get: 4321, GetNF: 9880, In: 1875977, Lock: 148141, NotOwner: 41243, Out: 1257708, PutNew: 231861, PutUpdate: 20518, Unlock: 169361
2025-02-20 12:00:00.000537 AData: 5, AIn: 1, ALock: 5, AOut: 1, ExpiredData: 232031, Get: 4288, GetNF: 9889, In: 1880423, Lock: 148571, NotOwner: 41461, Out: 1261121, PutNew: 232031, PutUpdate: 20623, Unlock: 169945
2025-02-20 13:00:00.000669 AData: 3, AIn: 2, ALock: 2, AOut: 1, ExpiredData: 231186, Get: 4286, GetNF: 9830, In: 1870289, Lock: 147772, NotOwner: 40896, Out: 1253897, PutNew: 231184, PutUpdate: 20476, Unlock: 168985
2025-02-20 14:00:00.004089 AData: 5, AIn: 1, ALock: 3, AOut: 1, ExpiredData: 229892, Get: 4067, GetNF: 9763, In: 1855035, Lock: 146815, NotOwner: 39594, Out: 1243690, PutNew: 229894, PutUpdate: 20324, Unlock: 167888
Clone this wiki locally