Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate between Nextcloud file system and objects #144

Merged
merged 38 commits into from
Jan 27, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
3a499a2
Add locking mechanism to ObjectEntity and ObjectService
rubenvdlinde Jan 15, 2025
aa1f2b7
Enhance ObjectEntityMapper and ObjectService with locking functionality
rubenvdlinde Jan 15, 2025
75eed8b
Why
rubenvdlinde Jan 15, 2025
f8784f4
Merge remote-tracking branch 'origin/development' into feature/CONNEC…
rubenvdlinde Jan 15, 2025
62c8e5f
Add object locking, unlocking, and reverting functionality
rubenvdlinde Jan 15, 2025
742f677
Add locking and ownership features to ObjectEntity and related compon…
rubenvdlinde Jan 15, 2025
1127873
Lets do migrations
rubenvdlinde Jan 15, 2025
e171711
First fixes on locking
rubenvdlinde Jan 16, 2025
f180efb
Add folder to register and object
rubenvdlinde Jan 16, 2025
9ee2d21
Merge branch 'development' into feature/CONNECTOR-179/locking
WilcoLouwerse Jan 16, 2025
67c24f5
Remove OpenConnector uses in OpenRegister ObjectService
WilcoLouwerse Jan 16, 2025
d7dac84
Add Folder to NC for any new object created
WilcoLouwerse Jan 16, 2025
3415b7d
Set folder property for Object & Register
WilcoLouwerse Jan 16, 2025
f7d1c33
Fix for setting folder property, do url instead of folder path
WilcoLouwerse Jan 16, 2025
64529c9
Merge branch 'development' into feature/CONNECTOR-179/locking
WilcoLouwerse Jan 17, 2025
dcb9be4
Added EventListeners for NextCloud Folders
WilcoLouwerse Jan 17, 2025
6d17d08
Added getter for the NC Node object of an object folder & Code cleanup
WilcoLouwerse Jan 17, 2025
fef17cb
Fetch file data from filesystem
rjzondervan Jan 20, 2025
e6cd5aa
Split getFiles function, add booleans to functions to enable the getF…
rjzondervan Jan 21, 2025
2dccbaf
Ga naar folder knop
rubenvdlinde Jan 21, 2025
99dbab3
Merge branch 'feature/CONNECTOR-179/locking' of https://github.com/Co…
rubenvdlinde Jan 21, 2025
4287b83
Make files visable in Open Register file detail view
rubenvdlinde Jan 21, 2025
e67bafa
format more like DCAT
rjzondervan Jan 21, 2025
62c6ef0
Merge remote-tracking branch 'origin/feature/CONNECTOR-179/locking' i…
rjzondervan Jan 21, 2025
8099abb
Documentation
rjzondervan Jan 21, 2025
41d7db6
Write file path instead of url in the folder property of an object
rjzondervan Jan 23, 2025
a7149a1
Create root folder separately and share with openregister group
rjzondervan Jan 23, 2025
ead73a6
Add docblock, remove unused function
rjzondervan Jan 24, 2025
3e6ff07
lint-fix
SudoThijn Jan 24, 2025
dbec7b9
Fixed disappearing lock button on unsuccessful call
remko48 Jan 27, 2025
750896f
Lint fix
remko48 Jan 27, 2025
30dcdfd
cleanup
remko48 Jan 27, 2025
0e2a61c
Spliting functions for getting files
rubenvdlinde Jan 27, 2025
e1cb264
Get proper url
rubenvdlinde Jan 27, 2025
5713a0c
Fix default lock duration
bbrands02 Jan 27, 2025
ec62064
Fix locked type to json
bbrands02 Jan 27, 2025
fad234b
Merge remote-tracking branch 'origin/feature/CONNECTOR-179/locking' i…
bbrands02 Jan 27, 2025
a5284d4
Lint fix
remko48 Jan 27, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion appinfo/info.xml
100755 → 100644
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Create a [bug report](https://github.com/OpenRegister/.github/issues/new/choose)

Create a [feature request](https://github.com/OpenRegister/.github/issues/new/choose)
]]></description>
<version>0.1.33</version>
<version>0.1.34</version>
<licence>agpl</licence>
<author mail="info@conduction.nl" homepage="https://www.conduction.nl/">Conduction</author>
<namespace>OpenRegister</namespace>
Expand Down
4 changes: 4 additions & 0 deletions appinfo/routes.php
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,15 @@
['name' => 'objects#mappings', 'url' => '/api/objects/mappings', 'verb' => 'GET'],
['name' => 'objects#auditTrails', 'url' => '/api/objects/audit-trails/{id}', 'verb' => 'GET', 'requirements' => ['id' => '[^/]+']],
['name' => 'objects#relations', 'url' => '/api/objects/relations/{id}', 'verb' => 'GET', 'requirements' => ['id' => '[^/]+']],
['name' => 'objects#files', 'url' => '/api/objects/files/{id}', 'verb' => 'GET', 'requirements' => ['id' => '[^/]+']],
['name' => 'schemas#upload', 'url' => '/api/schemas/upload', 'verb' => 'POST'],
['name' => 'schemas#uploadUpdate', 'url' => '/api/schemas/{id}/upload', 'verb' => 'PUT', 'requirements' => ['id' => '[^/]+']],
['name' => 'schemas#download', 'url' => '/api/schemas/{id}/download', 'verb' => 'GET', 'requirements' => ['id' => '[^/]+']],
['name' => 'registers#upload', 'url' => '/api/registers/upload', 'verb' => 'POST'],
['name' => 'registers#uploadUpdate', 'url' => '/api/registers/{id}/upload', 'verb' => 'PUT', 'requirements' => ['id' => '[^/]+']],
['name' => 'registers#download', 'url' => '/api/registers/{id}/download', 'verb' => 'GET', 'requirements' => ['id' => '[^/]+']],
['name' => 'objects#lock', 'url' => '/api/objects/{id}/lock', 'verb' => 'POST'],
['name' => 'objects#unlock', 'url' => '/api/objects/{id}/unlock', 'verb' => 'POST'],
['name' => 'objects#revert', 'url' => '/api/objects/{id}/revert', 'verb' => 'POST'],
],
];
98 changes: 98 additions & 0 deletions docs/object/object.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# Objects in OpenRegister

Objects are the core data entities in OpenRegister that store and manage structured information. This document explains everything you need to know about working with objects.

## Overview

An object in OpenRegister represents a single record of data that:
- Conforms to a defined schema
- Belongs to a specific register
- Has a unique UUID identifier
- Can contain nested objects and file attachments
- Maintains version history through audit logs
- Can be linked to other objects via relations

## Object Structure

Each object contains:

- `id`: Unique UUID identifier
- `uri`: Absolute URL to access the object
- `version`: Semantic version number (e.g. 1.0.0)
- `register`: The register this object belongs to
- `schema`: The schema this object must conform to
- `object`: The actual data payload as JSON
- `files`: Array of related file IDs
- `relations`: Array of related object IDs
- `textRepresentation`: Text representation of the object
- `locked`: Lock status and details
- `owner`: Nextcloud user that owns the object
- `authorization`: JSON object describing access permissions
- `updated`: Last modification timestamp
- `created`: Creation timestamp

## Key Features

### Schema Validation
- Objects are validated against their schema definition
- Supports both soft and hard validation modes
- Ensures data integrity and consistency

### Relations & Nesting
- Objects can reference other objects via UUIDs or URLs
- Supports nested object structures up to configured depth
- Maintains bidirectional relationship tracking

### Version Control
- Automatic version incrementing
- Full audit trail of changes
- Historical version access
- Ability to revert to any previous version
- Detailed change tracking between versions

### Access Control
- Object-level ownership
- Granular authorization rules
- Lock mechanism for concurrent access control

### File Attachments
- Support for file attachments
- Secure file storage integration
- File metadata tracking

## API Operations

Objects support standard CRUD operations:
- Create new objects
- Read existing objects
- Update object data
- Delete objects
- Search and filter objects
- Export object data
- Revert to previous versions

## Object Locking and Versioning

### Version Control and Revert Functionality

OpenRegister provides comprehensive version control capabilities:

- Every change creates a new version with full audit trail
- Changes are tracked at the field level
- Previous versions can be viewed and compared
- Objects can be reverted to any historical version
- Revert operation creates new version rather than overwriting
- Audit logs maintain full history of all changes including reverts
- Revert includes all object properties including relations and files
- Batch revert operations supported for related objects

### Locking Objects

Objects can be locked to prevent concurrent modifications. This is useful when:
- Long-running processes need exclusive access
- Multiple users/systems are working on the same object
- Ensuring data consistency during complex updates

#### Lock API Endpoints

To lock an object via the REST API:
201 changes: 197 additions & 4 deletions lib/Controller/ObjectsController.php
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,9 @@ public function __construct(
private readonly ContainerInterface $container,
private readonly ObjectEntityMapper $objectEntityMapper,
private readonly AuditTrailMapper $auditTrailMapper,
private readonly ObjectAuditLogMapper $objectAuditLogMapper
private readonly ObjectAuditLogMapper $objectAuditLogMapper,
private readonly ObjectService $objectService,

)
{
parent::__construct($appName, $request);
Expand Down Expand Up @@ -86,7 +88,8 @@ public function index(ObjectService $objectService, SearchService $searchService
$filters = $searchService->unsetSpecialQueryParams(filters: $filters);

// @todo: figure out how to use extend here
$results = $this->objectEntityMapper->findAll(filters: $filters);
$results = $objectService->getObjects(objectType: 'objectEntity', filters: $filters);
// $results = $this->objectEntityMapper->findAll(filters: $filters);

// We dont want to return the entity, but the object (and kant reley on the normal serilzier)
foreach ($results as $key => $result) {
Expand Down Expand Up @@ -160,6 +163,13 @@ public function create(ObjectService $objectService): JSONResponse
// Save the object
try {
$objectEntity = $objectService->saveObject(register: $data['register'], schema: $data['schema'], object: $object);

// Unlock the object after saving
try {
$this->objectEntityMapper->unlockObject($objectEntity->getId());
} catch (\Exception $e) {
// Ignore unlock errors since the save was successful
}
} catch (ValidationException $exception) {
$formatter = new ErrorFormatter();
return new JSONResponse(['message' => $exception->getMessage(), 'validationErrors' => $formatter->format($exception->getErrors())], 400);
Expand Down Expand Up @@ -206,6 +216,13 @@ public function update(int $id, ObjectService $objectService): JSONResponse
// save it
try {
$objectEntity = $objectService->saveObject(register: $data['register'], schema: $data['schema'], object: $data['object']);

// Unlock the object after saving
try {
$this->objectEntityMapper->unlockObject($objectEntity->getId());
} catch (\Exception $e) {
// Ignore unlock errors since the save was successful
}
} catch (ValidationException $exception) {
$formatter = new ErrorFormatter();
return new JSONResponse(['message' => $exception->getMessage(), 'validationErrors' => $formatter->format($exception->getErrors())], 400);
Expand Down Expand Up @@ -328,8 +345,6 @@ public function logs(int $id): JSONResponse
}
}



/**
* Retrieves all available mappings
*
Expand Down Expand Up @@ -380,4 +395,182 @@ public function getOpenConnectorMappingService(): ?\OCA\OpenConnector\Service\Ma

return null;
}

/**
* Lock an object
*
* @NoAdminRequired
* @NoCSRFRequired
*
* @param int $id The ID of the object to lock
* @return JSONResponse A JSON response containing the locked object
*/
public function lock(int $id): JSONResponse
{
try {
$data = $this->request->getParams();
$process = $data['process'] ?? null;
$duration = isset($data['duration']) ? (int)$data['duration'] : null;

$object = $this->objectEntityMapper->lockObject(
$id,
$process,
$duration
);

return new JSONResponse($object->getObjectArray());

} catch (DoesNotExistException $e) {
return new JSONResponse(['error' => 'Object not found'], 404);
} catch (NotAuthorizedException $e) {
return new JSONResponse(['error' => $e->getMessage()], 403);
} catch (LockedException $e) {
return new JSONResponse(['error' => $e->getMessage()], 423); // 423 Locked
} catch (\Exception $e) {
return new JSONResponse(['error' => $e->getMessage()], 500);
}
}

/**
* Unlock an object
*
* @NoAdminRequired
* @NoCSRFRequired
*
* @param int $id The ID of the object to unlock
* @return JSONResponse A JSON response containing the unlocked object
*/
public function unlock(int $id): JSONResponse
{
try {
$object = $this->objectEntityMapper->unlockObject($id);
return new JSONResponse($object->getObjectArray());

} catch (DoesNotExistException $e) {
return new JSONResponse(['error' => 'Object not found'], 404);
} catch (NotAuthorizedException $e) {
return new JSONResponse(['error' => $e->getMessage()], 403);
} catch (LockedException $e) {
return new JSONResponse(['error' => $e->getMessage()], 423);
} catch (\Exception $e) {
return new JSONResponse(['error' => $e->getMessage()], 500);
}
}

/**
* Revert an object to a previous state
*
* This endpoint allows reverting an object to a previous state based on different criteria:
* 1. DateTime - Revert to the state at a specific point in time
* 2. Audit Trail ID - Revert to the state after a specific audit trail entry
* 3. Semantic Version - Revert to a specific version of the object
*
* Request body should contain one of:
* - datetime: ISO 8601 datetime string (e.g., "2024-03-01T12:00:00Z")
* - auditTrailId: UUID of an audit trail entry
* - version: Semantic version string (e.g., "1.0.0")
*
* Optional parameters:
* - overwriteVersion: boolean (default: false) - If true, keeps the version number,
* if false, increments the patch version
*
* Example requests:
* ```json
* {
* "datetime": "2024-03-01T12:00:00Z"
* }
* ```
* ```json
* {
* "auditTrailId": "550e8400-e29b-41d4-a716-446655440000"
* }
* ```
* ```json
* {
* "version": "1.0.0",
* "overwriteVersion": true
* }
* ```
*
* @NoAdminRequired
* @NoCSRFRequired
*
* @param int $id The ID of the object to revert
* @return JSONResponse A JSON response containing the reverted object
* @throws NotFoundException If object not found
* @throws NotAuthorizedException If user not authorized
* @throws BadRequestException If no valid reversion point specified
* @throws LockedException If object is locked
*/
public function revert(int $id): JSONResponse
{
try {
$data = $this->request->getParams();

// Parse the revert point
$until = null;
if (isset($data['datetime'])) {
$until = new \DateTime($data['datetime']);
} elseif (isset($data['auditTrailId'])) {
$until = $data['auditTrailId'];
} elseif (isset($data['version'])) {
$until = $data['version'];
}

if ($until === null) {
return new JSONResponse(
['error' => 'Must specify either datetime, auditTrailId, or version'],
400
);
}

$overwriteVersion = $data['overwriteVersion'] ?? false;

// Get the reverted object using AuditTrailMapper instead
$revertedObject = $this->auditTrailMapper->revertObject(
$id,
$until,
$overwriteVersion
);

// Save the reverted object
$savedObject = $this->objectEntityMapper->update($revertedObject);

return new JSONResponse($savedObject->getObjectArray());

} catch (DoesNotExistException $e) {
return new JSONResponse(['error' => 'Object not found'], 404);
} catch (NotAuthorizedException $e) {
return new JSONResponse(['error' => $e->getMessage()], 403);
} catch (\Exception $e) {
return new JSONResponse(['error' => $e->getMessage()], 500);
}
}

/**
* Retrieves files associated with an object
*
* @NoAdminRequired
* @NoCSRFRequired
*
* @param string $id The ID of the object to get files for
* @return JSONResponse A JSON response containing the object's files
*/
public function files(string $id, ObjectService $objectService): JSONResponse
{
try {
// Get the object with files included
$object = $this->objectEntityMapper->find((int) $id);
$files = $objectService->getFiles($object);
$object = $objectService->hydrateFiles($object, $files);

// Return just the files array from the object
return new JSONResponse($object->getFiles());

} catch (DoesNotExistException $e) {
return new JSONResponse(['error' => 'Object not found'], 404);
} catch (\Exception $e) {
return new JSONResponse(['error' => $e->getMessage()], 500);
}
}
}
Loading
Loading