Most of the time series videos I have seen are done by so called experts , but themselves don’t know what they are teaching. Here at AI Engineering Srinivasan is doing a really great job. He knows to explain it to us by covering each and every corner of the topic. “If you can't explain it simply, you don't understand it well enough.” Albert Einstein
Unlike everyother idiot out there who predict single input LSTM's, you did a fantastic job in explaining how to create a samples for multiple inputs and multiple outputs. Just subscribed to your video, thank you!
This is very detailed explanation of windows, shifting and labeling in time series. I have came across Tensoflow Documentation, but I didn't understand at all. Thanks for sharing.
Thank you for the video, by far the best explanation I could find. Just one question, why do we add the column we are trying to predict in the data parameter of TimeseriesGenerator if we already have it in the target? I guess I'm having trouble understanding what those parameters are, but isn't data the set of features we want to use to predict the target?
Leon.. In time series future value is typically a functions of past lag of predictor. This is account for trend, seasonality and others. Reason in time series analysis we use lags of target. In traditional classification we typically do not use target as the output is just function of input feature alone
Hi... This is really a great content, at 20.28 min you mentioned features. I think they are samples. because we have None values in the last 2 rows of target , we have to skip their corresponding input samples. That is why we are considering all the samples except last 2. could you please check and confirm whether I am right or not ?.
nice explanation ...thank you..it will be more beneficial for beginners like us if you can share a notebook for learning purposes for us...thanks again...
Thanks for the video . It was very helpful . Still I have a doubt and a request . doubt :: When using lstm model at the time of prediction, to generate output of the second and subsequent timesteps , we will also need the other features at respective timesteps . So are you missing something in the video or is it my understanding that is wrong . Request :: when working with data of cryptocurrency price prediction , the range of values for price is very high . Using an ordinary scaler does not work and I suspect using log transform will induce a lot of error . Is there a way that can help . Can you make a video on it as it is on a related topic with a different set of problem . Thanks in Advance .
Normally, a model takes six consecutive time steps as input and predicts on 7th time step...now, I want the model to take six consecutive timesteps from t=0 to t=5 as input and t=12 as label, instead of t=7...what will be the arguments of TimeSeriesGenerator( ) in this case? Is it possible?
Anirban.. To my knowledge not possible out of the box. You can write a custom function and achieve it or I would say create a multi step output as shown in my example above that will predict all t-6 to t-12 and then pick t-12 as output
Anand.. Nope.. shift is how long of data you want to use to predict the future. So if length is 30 it will take past 30 days of data to predict 1 future. If you want future 30 days then it is like looping day by day with current prediction to get into future. Have you seen my LSTM time series video. I cover in that
@@AIEngineeringLife Thank you. So we also need to predict the other features ex: T_out, TH_1, Visibility. since we don't have it in the next days after the first prediction right?
Hi Srivatsan, thank you for the video. I have one question regarding Time Series using LSTM. I am working on a project where I have 3000 users, and corresponding to each user I have Time Series data. One naive thing would be to train a model for each user independently, but is there any other way I can train my LSTM to address this case? I would appreciate any input you can provide. Thanks
Best way is to train individual model for each time series but if you see the distribution is not different then you can check my multi time series DeepAR video where you can train a global model as well as individual models on each time series
I'm using RNN for my PG thesis work. I've a query. Do we have to run stationarity test for our time series data before feeding it in the neural network model... or this step is only required in traditional time series models like ARIMA?
Thank you for sharing this data preprocessing technique. I have a question: what if I am dealing with a CNN-LSTM case? In other words, my input features are in t*n*H*W shape. Can I still use this method? Hope for your answer, thanks!
I doubt this can be used. This is mostly for sequence structured data and might not work for image data where we have H and W. I maybe wrong as well in this case
Sir, Could pls explain how to treat time series classification if I have a feature with multiple values? The structure of the data is also given below label: 1 cl_data:10,10,2,12,12,12,12,12,1,1,1,1,1,3,3,3,3,3,1,1,1,1,1,1,1,1,1,1,3,3,1,1,1,1,1,3,3,3,3,3,3,1,1,1,1,1,1,3,3,3,1,1,1,1,1,1,1,1,1,1,3,3,3,3,3,3,1,1,1,1,1,1,1,4,4,4,4,2,2,1,1,1,1,1,1,2,1,1,1,3,3,3,3,1,1,1,1,10,10 label: 0 cl_Data:3,3,3,6,2,1,1,1,1,2,1,8,8,8,8,8,8,1,1,1,1,1,1,4,2,1,1,1,1,1,7,7,7,7,7,28,28,28,28,28,2,1,2,1,1,1,1,1,3,3,3,1,1,1,1,1,4,4,2,1,2,1,1,1,1,3,3,3,3,1,1,1,2,1,1,1,1,1
This is time series model and in this case future forecasts are dependent on past forecast in many cases by way of trend and seasonality. In time series typically target and feature might be similar but it takes lag of that variable rather learning one to one mapping
Sir, Would you mind to share some suggestion how to predict the following using LSTM please... 0, 1 and 2 are labels for the size of the network traffic. I want to use frame 1 and frame 2 to predict frame 0 This is the sequence of the frame 1 0 0 0 2 0 0 0 2 0 0 0 1....... Thank You
Did you try it as time series classification problem?.. basically create LSTM and then last layer define for multi class classification. Passing a sequence it must predict if it is 0,1 or 2
Sir, thank you so much for the valuable insight. I will try to frame it as a time series classification problem and will see. I have one question regarding time series prediction in the above case. For example, If i take 13 previous observation, the model can definitely predict the next point observation i.e 14th index as mentioned in your 1st Use Case. As far as the 2nd Use Case is concerned, Is it possible to use 13 previous observation to predict the data with ONLY label 0 ......OR do i need to take 13 previous observation and set offset as 13. But then it will predict the whole 13 sequence including 0, 1 and 2 data labels and looks like a multiple input and multiple output prediction problem. Therefore, I will appreciate your advice whether it is not possible to use data labels 1 and 2 to predict data label 0 as a time series prediction problem. If not, the only option is to use multi class time series classification. Thank You
No it is not.. The images are taken from TF website and not the code. If you see the same code in TF website then paste the link here. If you see my notebook on this the first 3 lines are images i download from TF website - github.com/srivatsan88/End-to-End-Time-Series/blob/master/Time_Series_Functions_for_Sequencing.ipynb Moreover it is official TF documentation and examples are given to use or teach. In work do we not use documentation?
