Skip to content

v0.1
Compare
Choose a tag to compare
@jason-zetaris jason-zetaris released this 22 Apr 06:23
· 101 commits to master since this release
v0.1
35b4b3e

Release Notes: Version 0.1

Date: April 22, 2024

Introduction:

Welcome to the latest release of our software! Version 0.1 brings initial featuers for data engineers, scientist and developers to run data catalog on top of Apache Spark.

Key Highlights:

Introducing data catalog on top of Apache Spark for the preparing data at any scale in ad-hoc analytics, data warehouse, lake house and ML project

New Features:

  • Multi level namespaces: User can create anonymous name space

  • Management of source system endpoint : Lightning catalog manages an endpoint of source system providing unified access to them through SQL and Apache SPARK API, which makes data discovery easy.

  • Query(SQL) capabilities : Lightning catalog allows to run ad-hoc query(SQL) over underlying heterogenous source systems in federate way, E,g Join between multiple source system without moving data.

  • Create virtual db schema : Lightning catalog allows to create virtual DB schema that keeps tables from different data sources

  • Supported data sources:
    DeltaLake
    Iceberg
    H2
    Snowflake
    Posstgres
    Oracle
    Mssql
    Redshift
    Terradata
    MySQL
    DB2
    SQLLite
    MariaDB
    Derby
    HANA
    Greenplum
    Vertica
    Netezza
    Csv
    Parquet
    Orc
    Json
    Avro

Additional Information:

For more information about this release, including detailed documentation and FAQs, please visit distribution web site,
https://www.zetaris.com/lightning-opensource
Document in git hub : https://github.com/zetaris/lightning-catalog/tree/master/doc

Assets 5
Loading