ABC Import: Import Your MongoDB, SQL, JSON, CSV Data Into Elasticsearch

We
have built abc import
to address the problem of indexing and syncing data from source X
into Elasticsearch (and appbase.io) via convenient CLI commands. It works today with MongoDB, Postgres, MySQL, SQL Server, JSON and CSV file formats.

abc import
is part of the abc
— A tool that allows accessing a
pp b
ase.io via a c
ommandline.

Why Use It

  1. Whether your data resides in Postgres or a JSON file or MongoDB or in all three places, abc
    can index the data into Elasticsearch. It is the only tool that allows working with all these sources at once or individually: csv
    , json
    , postgres
    , mysql
    , sqlserver
    , mongodb
    , elasticsearch
    .
  2. It can keep the Elasticsearch index synced in realtime with the data source. Note:
    This is currently only supported for MongoDB and Postgres.
  3. abc import
    is a single line command that allows doing all of the above. It doesn’t require any external dependencies, takes zero lines of code configuration, and runs as an isolated process with a minimal resource footprint.
  4. abc
    also supports configureable user defined transformations (think better ETL) for advanced uses to map data types, columns or transform the data itself before the data gets indexed into Elasticsearch.

See `abc import` in Action

Here, we show a realtime indexing example of using abc to sync MongoDB data with Elasticsearch.

You Should Use This If…

If you work with a SQL database and require a great search functionality but have been hesitating to use a search index because of the database ETL hell, abc
has got you covered.

abcis the only tool that supports all of these data formats today.

If you currently use application logic to write data to your source database and sync it with Elasticsearch, you should consider using abc
instead. Because it directly indexes from the database, it avoids a ton of false sync issues. By running the sync as a separate process, you create operational simplicity.

Get abc

Download
the binary of v0.4.1 release for MacOS, Linux or Windows. After that put the binary in a folder of your choice. You can put it in PATH so that it can be called from anywhere. Then test the installation as

> abc version

abcis also available as a docker image.

docker pull appbaseio/abc

Read the following guides for specific examples of using abc.

Postgres

CLI for Indexing data from Postgres to Elasticsearch

ElasticSearch is fantastic for indexing and filtering data. But hey, you have your data on a Postgres DB in production…
medium.appbase.io

Read the above article for how to sync a Postgres data source with Elasticsearch.

MongoDB

CLI for Indexing data from MongoDB to Elasticsearch

ElasticSearch is fantastic for indexing and filtering data. But hey, you have your data on a Mongo database in…
medium.appbase.io

Read the above article for how to sync a MongoDB data source with Elasticsearch.

MySQL

CLI for Indexing data from MySQL to Elasticsearch

ElasticSearch is fantastic for indexing and filtering data. But hey, you have your data in a MySQL database. How do you…
medium.appbase.io

Read the above article for how to sync a MySQL data source with Elasticsearch.

Microsoft SQL Server

CLI for Indexing data from MSSQL to Elasticsearch

ElasticSearch is fantastic for indexing and filtering data. But hey, you have your data in a MS SQL Server database…
medium.appbase.io

Read the above article for how to sync a MS SQL data source with Elasticsearch.

Elasticsearch

CLI for Indexing data from ElasticSearch to Elasticsearch

ElasticSearch is fantastic for indexing and filtering data. Won’t it be great if it is possible to sync data from one…
medium.appbase.io

Read the above article on how to sync Elasticsearch indices.

JSON

CLI for Indexing data from JSON to Elasticsearch

ElasticSearch is fantastic for indexing and filtering data. But hey, you have your data in a JSON file. How do you copy…
medium.appbase.io

Read the above article for how to sync a JSON data source with Elasticsearch.

CSV

CLI for Indexing data from CSV to Elasticsearch

ElasticSearch is fantastic for indexing and filtering data. But hey, you have your data in a CSV file. How do you copy…
medium.appbase.io

Read the above article for how to sync a CSV data source with Elasticsearch.

Summary

Some parts of abc
are open source and the source code is available on GitHub
. The program has been written in idiomatic Go. Show us your love by starring the repo. And try outappbase.io if you have not done so already, it’s a super easy service to have your own streaming NoSQL database on the cloud.

稿源:All things #datastreams (源链) | 关于 | 阅读提示

本站遵循[CC BY-NC-SA 4.0]。如您有版权、意见投诉等问题,请通过eMail联系我们处理。
酷辣虫 » 后端存储 » ABC Import: Import Your MongoDB, SQL, JSON, CSV Data Into Elasticsearch

喜欢 (0)or分享给?

专业 x 专注 x 聚合 x 分享 CC BY-NC-SA 4.0

使用声明 | 英豪名录