Sir, you are a saint! Thank you thank you thank you!!! Not only did you make this easy but you gave me peace of mind. If we ever meet in person, I hope you will give me the honor of buying you a drink.
Fantastic video! The only thing that would have been nice to see is how do you take these clustering solutions and create a new column with the cluster number for each observation that could then be exported to Excel and used to create a PPT slide. This would have really been helpful for work situations.
This is the best video I have watched about cluster analysis. I subscribed immediately after watching this. Here's my question. I don't know if I skipped this in the video but how do I extract the vector containing the specific cluster each observation belongs to? I tried doing the model-based method.
I didn't show this in the video sorry, but you can extract the vector of most probable clusters by accessing the $classification component of your mclust model. If you want more detailed information, i.e. the matrix of probabilities for each datum belonging to each class, extract the $z component. In R, it's always useful to call str() on your model objects so you can understand and inspect their structure. Using ?Mclust and reading the Value section, also shows you what each component of the model object means.
@@hefinrhys8572 oh okay thanks. I didn't think of using ?Mclust. I'm going to use your videos as reference. Very informative and understandable. Thanks again.
Hallo Hefin, wieder ein großartiges Tutorial. Erlaube eine Frage: Wie kann ich zu einem Wert (oder Wertepaar x, y) aus dem Datensatz das zutreffende Cluster zuordnen? Eindimensionales Beispiel: Werte 1 - 10 = Cluster 1 11 - 20 = Cluster 2 21 - 30 = Cluster 3 Zu welchem Cluster gehört Var = 19? Antwort: Var (=19) gehört zum Cluster 2. Wie berechne ich das mit kmeans bzw. R (ggplot2)? Hello Hefin, another great tutorial. Allow a question: How can I assign the appropriate cluster to a value (or value pair x, y) from the data set? One-dimensional example: values 1 - 10 = cluster 1 11 - 20 = cluster 2 21 - 30 = cluster 3 Which cluster does Var = 19 belong to? Answer: Var (= 19) belongs to cluster 2. How do I calculate this with kmeans or R (ggplot2)?
Thank you for the information, please help me to understand how the results of PCA is used in clustering, because both the videos of PCA and Clustering are found not continued, please help us if we want to continue clustering with results of PCA how we should do that..again thank you for detailed information
If you wish to cluster the results of a PCA, simply select the number of principal components you wish to retain (the first few that explain most of the variance), and use the data projected onto these components as the input data to your clustering algorithm. But I would suggest you compare the clusters based on the original data, and the clusters based on the reduced dimensional data, to see which performs better.
Excellent study material. BUT BECAME BLURRED FROM 6.45 TO ABOUT 16.05 MINUTES. Please how can I study the entire video clearly? Thank you for such a summary.
Is there a clustering method indicated to a certain number of variables.? I have in my study 32 variables and I'm thinking that perhaps it would be a specific procedure to that much of variables. thanks for sharing you knowledge.
Thank you for the wonderful tutorial!!! I have one question. Is there anyway to create different data frames based on the clusters identified? For instance, having a new column for 'cluster' and have the cluster # for each row. I am doing the following but not sure if it is right. kc
Nice spot. Using usual definition of BIC yes, lower is better. The mclust package rearranges the equation to be: BIC = L - 0.5 * p * ln(n) Such that the expression should be maximised instead of minimised.
@@hefinrhys8572 Ok i didn't know that definition , maybe they need to not call it BIC to not confuse the people. This part is for the people that read this reply and maybe don't know what we are talking: BIC=k ln(n)-2ln(L), k is the number of parameter estimated by the model, L is the maximized value of the likelihood function of the model and n the number of observations (sample size). Lower BIC is, less information we lose.
hi, i keep getting the error: " Error in xy.coords(x, y, xlabel, ylabel, log) : 'x' and 'y' lengths differ" when i try and plot the graph to see the number of k with this code - plot(1:10, betweenss_totss, type = "b", + ylab = "Between SS / Total SS", xlab = "Clusters (k)") Do you know how i can solve this? I have tried looking online and have found how to make the x and y axis the same, however for this graph we don't need them to be the same. Any information you could spare would be great!!
Hi Sophie, the first two arguments to the plot() function need to be vectors representing the x and y axes, respectively. At the moment, you only supply a vector for the x axis (1:10) that only contains values 1 through 10. What are you trying to plot here? The easiest way to plot the clusters is to create a new column in the data.frame indicating cluster membership, plotting two variables against each other and colouring by this cluster variable.
Thank you! Sorry for the late reply. So irisScaled is a matrix (which I think I omitted to mention in the video), and a succinct way to get the means of each column would be to use the apply() function: apply(irisScaled, 2, mean) where the first argument is the data, the second is an index (either 1 to iterate over rows or 2 for columns) and the third argument is the function to apply. You could also find the mean for each column individually like this: mean(irisScaled[, 1])
You're very welcome. I'm sorry for the late reply to this; I'm not familiar with theory of planned behaviour models, but if you describe the problem you're trying to solve, I may be able to help.
I am just curious, how would I go about removing a trend in my data and then using all the observation into clustering. I have several countries and several variables but observed over time [annually]. The bad thing is that picking a year to do clustering is an option but not that much great since explanation can be fault if its based only on one year - if I am not mistaken xD
Hi Wildfox, sorry for the late reply. So I presume you are looking for clusters of countries? One approach would be to model the relationship between time and a dependent variable (if you have one) for each country using a linear model. Then, use the estimated marginal means (also called least squares means) to cluster the countries. Estimated marginal means are the predicted means of variables in a linear model after accounting for all the other variables (i.e. after removing the effect of time).
Honestly, it depends on what you are clustering. But there is the risk of too many variables which will cause problems clustering, I think this falls down to trial and error also using domain knowledge of what variables to include.
Sorry for the slow reply to this. I hope you worked it out, but I would extract the assigned groups using something like this: # ADD NEW COLUMN TO DATAFRAME WITH MCLUST GROUP ASSIGNMENTS irisScaled$Group
Hi Nikhil, I'm not quite sure I understand what you are trying to do. Do you mean defect clustering in application testing? This isn't a statistical computing application per se, but here is some information which may be of use: www.pitsolutions.ch/blog/defect-clustering-and-pesticide-paradox/
Hi ghdoia, so R and its packages come with datasets built in. To list them all, simply run data(), to load one (such as the iris dataset), run data(iris). Then, you can access the iris dataset by referring to it by name. I hope that helps.
@@hefinrhys8572 thank you very much. I learnt alots from you. I've just had some of issue when I did run the below code : plot(1:10, betweenss_totalss, type = "b", ylab = "Between SS/Total SS", xlab = "Cluster(K)") as it shows this message below; Error in plot.window(...) : need finite 'ylim' values In addition: Warning messages: 1: In min(x) : no non-missing arguments to min; returning Inf 2: In max(x) : no non-missing arguments to max; returning -Inf Please can you tell what is that mean and how i can solve this problem?
Hi Priyanwada, I'm reading this on my phone so cannot check in R right now, but it looks like you have 'list()' in front of 'for(i in 10)'. This 'list()' function is empty and I'm not sure you need it anyway.
