Python time series database

tsdb 0.0.9

TSDB is created to help researchers and engineers get rid of data collecting and downloading, and focus back on data processing details. TSDB provides all-in-one-stop convenience for downloading and loading open-source time-series datasets (available datasets listed below).

❗️Please note that due to people have very different requirements for data processing, data-loading functions in TSDB only contain the most general steps (e.g. removing invalid samples) and won’t process the data (not even normalize it). So, no worries, TSDB won’t affect your data preprocessing. If you only want the raw datasets, TSDB can help you download and save raw datasets as well (take a look at Usage Examples below).

🤝 If you need TSDB to integrate an open-source dataset or want to add it into TSDB yourself, please feel free to request for it by creating an issue or make a PR to merge your code.

🤗 Please star this repo to help others notice TSDB if you think it is a useful toolkit. Please properly cite TSDB in your publications if it helps with your research. This really means a lot to our open-source research. Thank you!

❖ Usage Examples

TSDB now is available on ❗️

Install it with conda install tsdb , you may need to specify the channel with option -c conda-forge

or install from source code:

pip install https://github.com/WenjieDu/TSDB/archive/main.zip

That’s all. Simple and efficient. Enjoy it! 😃

❖ List of Available Datasets

Name	Main Tasks
PhysioNet Challenge 2012	Classification, Forecasting, Imputation
PhysioNet Challenge 2019	Classification, Imputation
Beijing Multi-Site Air-Quality	Forecasting, Imputation
Electricity Load Diagrams	Forecasting, Imputation
UCR & UEA Datasets (all 160 datasets)	Classification
Vessel AIS	Classification, Forecasting, Imputation

❖ Citing TSDB/PyPOTS

TSDB is a part of PyPOTS project (a Python toolbox for data mining on Partially-Observed Time Series), and was separated from PyPOTS for decoupling datasets from learning algorithms.

The paper introducing PyPOTS project is available on arXiv at this URL, and we are pursuing to publish it in prestigious academic venues, e.g. JMLR (track for Machine Learning Open Source Software). If you use TSDB in your work, please cite PyPOTS project as below and 🌟star this repository to make others notice this library. 🤗 Thank you!

Источник

How to collect and process time-series data using Redis and Python

Time series data is a series of data stored in the time order (Example: Stock performance over time). Industries today are collecting and analyzing time-based data more than ever before. Traditional databases that rely on relational or document data models are designed neither for storing and indexing data based on time, nor for running time-bucketed aggregation queries. Time-series databases fill this void by providing a data model that optimizes data indexing and querying by time.

Time Series is a Redis feature, It allows Redis to be used as a fast in-memory time series database designed to collect, manage, and deliver time series data at scale. The Time Series feature shares the performance and simplicity aspects of Redis. Under the hood, it uses efficient data structures such as Radix tree to index data by timestamp, which makes it extremely fast and efficient to run time-aggregate queries.

Python Client for Time Series with Redis​

As of redis-py 4.0.0, the redistimeseries-py library is deprecated. It’s features have been merged into redis-py. Please either install it from pypy or the repo.

Follow the steps below to get started with Time Series with Redis in Python:

Step 1. Create a free Cloud account​

Create your free Redis Enterprise Cloud account. Once you click on “Get Started”, you will receive an email with a link to activate your account and complete your signup process.

For a limited time, use TIGER200 to get $200 credits on Redis Enterprise Cloud and try all the advanced capabilities!

Step 2. Create Your database​

Choose your preferred cloud vendor. Select the region and then click «Let’s start free» to create your free database automatically.

If you want to create a custom database with your preferred name and type of redis, click «Create a custom database» option shown in the image.

Step 3. Verify the database details​

You will be provided with Public endpoint URL and «Redis Stack» as the type of database with the list of features that comes by default.

Step 4.Installation​

Step 5. Create a script file​

 import redis  r = redis.Redis(host='redis-18386.c110.qa.us-east-1-1.ec2.qa-cloud.redislabs.com', port=add port>, password=add password>)  r.ts().create(2, retension_msecs=5) 

Save the above file with a name «ts.py».

Step 6. Executing the python script​

Step 7. Monitor the Redis database​

 1648389303.557366 [0 20.127.62.215:59768] "TS.CREATE" "2" 

References​

• Getting Started with Time Series with Redis
• Learn more about Time Series with Redis in the QuickStart tutorial.
• Build Your Financial Application on Time Series with Redis
• How to Manage Real-Time IoT Sensor Data in Redis
• Introduction to Time Series with Redis — Video

Источник