gitmyhub

dinky

Java ★ 3.7k updated 2mo ago

Dinky is a real-time data development platform based on Apache Flink, enabling agile data development, deployment and operation.

A browser-based platform for writing, testing, and managing Apache Flink SQL streaming jobs, with a built-in editor, debug previews, job monitoring, savepoints, and connections to a wide range of databases.

JavaApache FlinkSpring BootMyBatis PlusMonaco Editorsetup: hardcomplexity 4/5

Dinky is a browser-based platform for writing and running SQL jobs on Apache Flink, a system for processing streams of data in real time. If your company needs to move, transform, or analyze data as it flows from one system to another, Flink does that processing, and Dinky gives teams a web interface to write and manage those jobs without working entirely from the command line.

The built-in editor includes autocomplete, syntax highlighting, and a debug preview that lets you see what a query's results look like before the job runs at full scale. Queries can be submitted to a range of deployment environments, including local machines, standalone clusters, Yarn-managed clusters, and Kubernetes clusters, depending on what infrastructure your team already runs. A syntax checker and a plan viewer show how a query will be executed step by step before anything is committed.

Beyond writing SQL, the platform includes an operations dashboard where teams can monitor running jobs, view logs, take savepoints (snapshots that preserve the state of a long-running job), configure alerts, and inspect data lineage, which shows which fields in your output came from which source tables. Dinky connects to a wide range of external databases and systems, including ClickHouse, Doris, Hive, MySQL, Oracle, PostgreSQL, and SQL Server, so it can sit at the center of a mixed data environment.

The project is open source under the Apache 2.0 license and written primarily in Java. It is built on top of Apache Flink and relies on other open-source libraries including Spring Boot, MyBatis Plus, and Monaco Editor, which is the same code editor that powers VS Code. The README is written mostly in Chinese, so the primary community and documentation are Chinese-language, though an English README is also linked. The planned roadmap includes multi-tenant support, global lineage analysis, and unified metadata management.

Where it fits