Hidden Secrets about Instrumenting JVMs for OpenTelemetry

A presentation at JNation 2021 in June 2021 in by Ricardo Ferreira

Slide 1

Slide 1

Hidden Secrets about Instrumenting JVMs for OpenTelemetry Ricardo Ferreira @riferrei

Slide 2

Slide 2

OpenTelemetry Crash Source @riferrei

Slide 3

Slide 3

opentelemetry overview @riferrei

Slide 4

Slide 4

Quick Demo https://github.com/riferrei/otel-with-java @riferrei

Slide 5

Slide 5

Much better now đź‘Ť @riferrei

Slide 6

Slide 6

Ricardo Ferreira Developer Advocate q Elastic Community Team q HashiCorp Ambassador q Before working for Elastic: Confluent, Oracle, Red Hat q Distributed Systems, O11y, Streaming Systems, databases q riferrei@elastic.co q riferrei@riferrei.com @riferrei

Slide 7

Slide 7

JVM Instrumentation options java -javaagent:./${OTEL_AGENT} \ -Dotel.traces.exporter=otlp \ -Dotel.metrics.exporter=otlp \ -jar target/app.jar 80% @riferrei API, SDK, SDK-Extensions 20%

Slide 8

Slide 8

Manual Instrumentation @riferrei

Slide 9

Slide 9

Manual Instrumentation <dependency> <groupId>io.opentelemetry</groupId> <artifactId>opentelemetry-sdk-extension-autoconfigure</artifactId> <version>${version}</version> </dependency> @riferrei

Slide 10

Slide 10

Down the river via @WithSpan <dependency> <groupId>io.opentelemetry</groupId> <artifactId>opentelemetry-extension-annotations</artifactId> <version>${version}</version> </dependency> @WithSpan public void someMethod() { } @riferrei

Slide 11

Slide 11

Disabling auto instrumentation -Dotel.instrumentation.common.default-enabled=false @riferrei

Slide 12

Slide 12

Enabling only manual instrumentation -Dotel.instrumentation.opentelemetry-annotations.enabled=true @riferrei

Slide 13

Slide 13

Meaningful names for services -Dotel.resource.attributes=service.name= \ brand-estimator,service.version=1.0 @riferrei

Slide 14

Slide 14

Meaningful names for services export OTEL_RESOURCE_ATTRIBUTES=service.name= \ brand-estimator,service.version=1.0 @riferrei

Slide 15

Slide 15

Making your traces observable @riferrei

Slide 16

Slide 16

Using Text-Map-Propagators @riferrei

Slide 17

Slide 17

Using Text-Map-Propagators @riferrei

Slide 18

Slide 18

Using Text-Map-Propagators @riferrei

Slide 19

Slide 19

Can I tell you a secret? SELECT a FROM b WHERE password=? SELECT a FROM b WHERE password=”secret” @riferrei

Slide 20

Slide 20

Databases can be a bit chatty java.sql.Datasource#getConnection() -Dotel.instrumentation.jdbc-datasource.enabled=true @riferrei

Slide 21

Slide 21

Excluding noisy frameworks @riferrei

Slide 22

Slide 22

Using the batch-Span-Processor @riferrei

Slide 23

Slide 23

Planning your sampling strategy @riferrei

Slide 24

Slide 24

Debugging the Agent -Dotel.javaagent.debug=true export OTEL_JAVAAGENT_DEBUG=true @riferrei

Slide 25

Slide 25

Disabling the agent -Dotel.javaagent.enabled=false export OTEL_JAVAAGENT_ENABLED=false @riferrei

Slide 26

Slide 26

Observable metrics: no pun intended @riferrei

Slide 27

Slide 27

Inspecting telemetry data with jMC <dependency> <groupId>io.opentelemetry</groupId> <artifactId>opentelemetry-sdk-extension-jfr-events</artifactId> <version>${version}</version> </dependency> @riferrei

Slide 28

Slide 28

THANK YOU Ricardo Ferreira @riferrei