Thanks for Great session. I have few questions. What is the benefit of using opentelemetry over normal spring boot default metrics? We have a batch auto scaling application and We are doing analysis if We can use otel
I have downloaded the code yesterday. I have started the items and users services on my local. I have also run docker compose, so all the apps are running, no problem. I see the items and users app are started with the 8081 and 8091 for prometheus to scrape metrics from. My question is, I don't see any app started on 8081 or 8092. Also, I see on the video when you trigger localhost:8081/metrics you can see metrics displayed, on my local it shows no connection found. Did you start explicitly any app listening on 8081 and 8092 ?
00:00 Introduction to OpenTelemetry and its purpose 02:22 Expert in microservices and open telemetry, offering free boot camp 07:19 Live debugging and troubleshooting process 09:42 Troubleshooting service A's inability to write to db1 14:15 The importance of having logs, metrics, and traces for resolving issues quickly. 16:46 OpenTelemetry is about logs, metrics, and traces for observability. 21:15 OpenTelemetry collects logs, metrics, and traces for applications. 23:23 Introduction to OpenTelemetry and its Implementation 27:55 OpenTelemetry helps construct the structure of a trace. 30:01 Introduction to OpenTelemetry implementation 34:39 Improving Moki UI performance can enhance overall trace performance and latency. 36:55 Setting up metric configuration and Prometheus exporter 41:25 Attaching trace ID to logs for effective troubleshooting. 43:28 OpenTelemetry enables automatic tracing and logging of API calls. 47:48 Troubleshooting failures and achieving goals in OpenTelemetry deployment 49:58 OpenTelemetry is about collecting and distributing data, not just data visualization. 54:04 Promoting OpenTelemetry to the community Highlights of this video for me :) Amazing !!
can you please make a video on collecting infra level metrics & logs from kubernetes cluster using opentelemetry using deamonset mode (without doing instrumentation to application/microservices). And View those metrics & logs in grafana and store those metrics & logs in Storage Accounts &/ Database. Is it possible?
Thank you for the video, really helpful! One question I had was why doesn't Jaeger show different colors for different services : for example different colors for "otel-troubleshooting" and "redis" services?
i am getting this app errors while running the docker-compose . (artstore) robeshranjan@robeshranjan-mac user % docker logs todo-user Error: No such container: todo-user (artstore) robeshranjan@robeshranjan-mac user % docker logs 9229c266ea62 Traceback (most recent call last): File "/usr/local/bin/opentelemetry-instrument", line 8, in <module> sys.exit(run()) File "/usr/local/lib/python3.9/site-packages/opentelemetry/instrumentation/auto_instrumentation/__init__.py", line 115, in run execl(executable, executable, *args.command_args) File "/usr/lib64/python3.9/os.py", line 542, in execl execv(file, args) TypeError: execv: path should be string, bytes or os.PathLike, not NoneType (artstore) robeshranjan@robeshranjan-mac user %
i am using java, and apart from context propagation, all bits (that i need to start with) are automatically instrumented.... how do i configure context propagation for rest requests, rabbit messages, etc?
I thought that one of the important architectural characteristics of Prometheus is that it collects data from the services that it monitors. By having the Prometheus exporter is it providing a new location to scrape or is it pushing data to Prometheus?
I have a naive question about that why you need to use a col-agent and a col-gateway? I mean it's ok to use only one otel-collector to receive, process, and then export the span data to jaeger-collecor then store into ES? Meanwhile, another question is that in your scenario, if the jaeger-agent component is unnecessary because your endpoint is "jaeger:14250" which means your directly connect to jaeger-collector?
Hi! so for the first question, you can use a single collector, there is nothing wrong with that. Yes, you don't have to user the jaeger-agent you can work directly with the collector
Hi. can I export open telemetry data diretly to mongodb or some another data base and configure jaeger to get data from it. Do we have any direct exporters we can use to export data from OpenTelemetry collector to DB?
Yes of course. There are several databases you can export to. All the exporters can be found in the OpenTelemetry contrib repo: github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter
I liked the bootcamp and really benefited from it. I would like to add a note at 26:53 , OTLP expoter now supports telemetry signal "logs" , we would need to use otel-collector >0.61.0
Thanks for the information. This is really helpful. I have one query. I came across one issue in OTEL collector pod for DataDog Exporter. I am not able to set a proxy for DataDog exporter as the Data Dog site is only accessible through proxy. Do you have any idea. Your help will be highly appreciated. Thanks.
Love the presentation on open telemetry, thanks for sharing. I have a question regarding the following scenario. Is there a possibility to connect the trace is and span id with information coming from other sources. For example I run function x with input this and output that for which I have a span generated that I will add in the logs with structured logging and would like to cross reference it with metrics that were collected by kubernetes, or other sources to be able to identify which function generated an unusual spike in my memory graph. I would like to be able to navigate between graphs and see different aspects of the same problem for root cause analysis. A second question would be, if I use hibernate and use a span for it, can I somehow connect the transaction on the hibernate side in Java, through a unique id to a database transaction, so that in case I have issues with the call to be able to investigate (database in the beginning, networking maybe afterwards). Last question as to do with the response to an http(s) request. I would like to be able to capture the end time, packet received and success or failure as part of this in an application as part of the end to end flow. Can it call back somewhere to the gateway as the end step of the flow to finalize the validation of the package arrival?
Hi Alex, thanks for the kind words. For your first question, OpenTelemetry offers to correlate an example of trace to a metric, you can read about it more here: opentelemetry.io/docs/reference/specification/metrics/data-model/ For your second question: Yes, you can add to the span some unique id as an attribute, and that will help you to correlate. And for your last question: I'm not sure I got this one, but the HTTP response will automatically have latency, HTTP code etc...
Regarding the third one I would like to be able to send back to the same data source the result on the front-end side. When the response was received by the client (calculate cycle time between gateway and client), was it a 200 response, what was the payload. In case of a failure not only the code but maybe the root case (similar to the ones done in unit testing, expected this but got that). I believe such a setup (of course taking into consideration the sampling where the trace is stable) could be used for multiple aspects to characterize the service connecting it to SLA''s at multiple levels and isolating parts of the system that seem to have issues. Later on, you can group such traces into multiple business functions and understand how they each perform at all levels in the company and understand the system constraints. Might be able at some point in time to connect to some financial metrics too to re-architect the solution. (seen lately a company move their CDN from the cloud to a local solution that produced significant changes to the cost of the sevice). Thanks again for the presentation and the information provided, very valuable.
@0:30 How is a collector different from an agent? I still need to run a collector to get my telemetry; wether I call it an agent or collector seems no difference to me?
You are correct Rob! They are both images for the collector. The agent is supposed to run near the application and serve as its own private collector. The OpenTelemetry collector is the primary collector that receives all the spans from all the apps.
