On The Suitability of Differential Dataflow For Datalog Interpretation In Highly Dynamic Settings

Abstract:In the domain of knowledge representation and reasoning within AI, datalog engines play an ever-increasingly crucial role. The crux of their operation lies in materialization: the evaluation of a data- log program and its incorporation into a database. This operation becomes complex and resource-intensive, especially when the data is highly dynamic, as it is common in distributed environments. Thus, incremental materialization, adjusting the computation to new data instead of restarting it, is the norm. However, handling the deletion of data is significantly more complicated than addition due to the cascading effects of what is being removed. Differential Dataflow offers a computational model that effectively addresses this, ensuring consistent performance for both data additions and deletions. In this paper, we delve into the efficiency of materializa- tion using three distinct datalog implementations: one based on a streamlined relational engine and two others that implement the same algorithm, but with one utilizing differential-dataflow, and another not. Our insights provide a roadmap for enhancing datalog- driven computations, particularly in dynamic data environments like the cloud.

Databases (cs.DB); Artificial Intelligence (cs.AI); Programming Languages (cs.PL)

I.2.1; I.2.4; I.2.5

arXiv:2308.04214 [cs.DB]

(or arXiv:2308.04214v2 [cs.DB] for this version)

Proceedings of the 2023 6th Artificial Intelligence and Cloud Computing Conference (AICCC 2023), Kyoto, Japan, December 2023, pp. 218-225. ACM, New York, NY, USA

https://doi.org/10.1145/3639592.3639622 DOI(s) linking to related resources