microsoft azure storage explorer compatibility

#Microsoft azure storage explorer compatibility how to

However, for S3, there are additional caveats on concurrent writes. Delta Lake uses the scheme of the path (that is, s3a in s3a://path) to dynamically identify the storage system and use the corresponding LogStore implementation that provides the transactional guarantees.

Storage systems with built-in support: For some storage systems, you do not need additional configurations. There are two categories of storage systems:

#Microsoft azure storage explorer compatibility how to

This article covers how to configure Delta Lake for various storage systems. To provide the ACID guarantees for different storage systems, you may have to use different LogStore implementations. Mutual exclusion: Only one writer must be able to create (or rename) a file at the final destination.Ĭonsistent listing: Once a file has been written in a directory, all future listings for that directory must return that file.īecause storage systems do not necessarily provide all of these guarantees out-of-the-box, Delta Lake transactional operations typically go through the LogStore API instead of accessing the storage system directly. Specifically, Delta Lake relies on the following when interacting with storage systems:Ītomic visibility: There must a way for a file to visible in its entirety or not visible at all. Access Delta tables from external data processing enginesĭelta Lake ACID guarantees are predicated on the atomicity and durability guarantees of the storage system.Production Configuration (S3 multi-cluster).