Тёмный
No video :(

Interacting with Elasticsearch via Python - A Primer in 2021 

H & A Security Solutions
Подписаться 4,9 тыс.
Просмотров 11 тыс.
50% 1

Learn how to interact directly with Elasticsearch using Python as well as found out some use cases for doing so. This video explores using Python against Elasticsearch based on a project to implement custom lifecycle management strategies for index deletion, rollovers, allocation, and cloud accounting. H & A Security Solutions is sharing some of our experiences, both good and bad, with Elasticsearch and how Python can lift barriers you come up against.
Link to Elastic ILM project used during the video can be found here:
github.com/HAS...
2:59 Official Elasticsearch Python documentation (elasticsearch-...)
3:20 Roberto Rodriguez Jupyter Notebooks from SIGMA Rules to Query Elasticsearch ( / jupyter-notebooks-from... )
Do not forget to subscribe to this channel for updates on future videos. Also, H & A Security Solutions provides aid to clients on a daily basis to use, implement, and maintain Elastic stacks as a SIEM.
Looking for help improving your detection capabilities? Reach out to info@hasecuritysolutions.com or (217) 730-3007. We routinely implement SIEM and NSM solutions both commercial and open source.
Speaker: Justin Henderson, CEO H & A Security Solutions LLC
Justin is the co-founder of H&A Security Solutions, LLC, a company that deploys, maintains, and tunes SIEM, NSM, and other solutions for organizations. Justin also maintains one of the largest security onion deployments in the world with over 1200 network sensors. He is a passionate security architect and researcher whose experience in cybersecurity started at the age of thirteen when he began providing professional services to organizations. Justin was the 13th GSE to become both a red and blue SANS Cyber Guardian and holds over 60 industry certifications. As the author of SEC555 and co-author of SEC455 and SEC530, he’s able to bring his encyclopedia of IT knowledge into the classroom.
Follow Justin on Twitter @SecurityMapper at / securitymapper or using LinkedIn at / justinhenderson2014 .

Опубликовано:

 

26 авг 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 26   
@tobypass108
@tobypass108 8 месяцев назад
Love this video. I was able to follow along. Rather than just using your repo, I downloaded it with git then coped the relevant functions and log files from es.py into my script so I could see how it worked. Really good walkthrough - appreciate the knowledge sharing. Thank you.
@adliwahid6081
@adliwahid6081 3 года назад
Thanks for this! Been putting off ILM for too long - the video with code really helps demystify certain things - cheers!
@HASecuritySolutions
@HASecuritySolutions 3 года назад
Glad it helped!
@kevinz1991
@kevinz1991 2 года назад
Fantastic video with extremely clear explanation. Fantastic teacher thank you
@josemariaaresmartinez8071
@josemariaaresmartinez8071 2 года назад
Awesome video, thanks for sharing :)
@HASecuritySolutions
@HASecuritySolutions 2 года назад
Thank you
@IvarsRuza
@IvarsRuza 3 года назад
wow U saved my headache - enrichment after the index is set. I was doing it in logstash - but thats a lot of planing - I do always forget smtg and had rebuild the index from syslog again and again.
@HASecuritySolutions
@HASecuritySolutions 3 года назад
I still love logstash but it definitely requires planning. Knowing how to do things directly with data in Elasticsearch opens a lot of doors
@pismed2495
@pismed2495 8 месяцев назад
can you help me i will connect to elastic cloud with file python
@riyanirawan256
@riyanirawan256 2 года назад
hello, I want to ask. I have executed the query and in the index data I entered. but the csv/json file is not uploaded to the file. I checked the index pattern, the health status is yellow and the file size is 208b???
@HASecuritySolutions
@HASecuritySolutions 2 года назад
It did not bring in the data then. It probably also shows 0 documents. Most likely what you are experiencing is the changes in the elasticsearch-py library since this repo was put together. If you flip to the opensearch-py library it should still work as in this video. Moving forward, elasticsearch-py has some changes to how it operates. I may try doing an updated video and github repo later to show the differences.
@TheMiguel710
@TheMiguel710 2 года назад
Can I import 2 schemas from elasticsearch to dataframes and perform joins on the jupyter notebook?
@HASecuritySolutions
@HASecuritySolutions 2 года назад
Can you clarify what you are trying to do? I believe the answer is yes. You can query various index data into dataframes and then perform join operations against them
@TheMiguel710
@TheMiguel710 2 года назад
@@HASecuritySolutions I want to import two indexes from elasticsearch to the notebook as separate dataframes and join them to cross check information
@lauragalera6826
@lauragalera6826 3 года назад
pretty useful. However, is there a way that, when you transform the response to a dataframe, to add a column _score? Seems the response does not include it tho...
@HASecuritySolutions
@HASecuritySolutions 3 года назад
Elasticsearch responses won't natively be a dataframe. However, you can use pandas or any other library of choice to convert the responses into one
@mp3311
@mp3311 2 года назад
Thank you. How could I access the elastic search records for a given period of time for a given index?
@HASecuritySolutions
@HASecuritySolutions 2 года назад
For this, use the same search in the video but at a range query to it
@daviddonadze221
@daviddonadze221 3 года назад
Thank you so much.
@HASecuritySolutions
@HASecuritySolutions 3 года назад
You're welcome!
@pulkitdikshit6148
@pulkitdikshit6148 2 года назад
can we create index without using elastic search python module? by using endpoint url and how to bulk upload data on those created index?
@HASecuritySolutions
@HASecuritySolutions 2 года назад
Yes you can. For that you simply submit calls to Elasticsearch based on it's API. They have lots of examples in their documentation using curl. If you are using OpenSearch, they have their own python module that is nearly identical
@pulkitdikshit6148
@pulkitdikshit6148 2 года назад
@@HASecuritySolutions okay, thanks for answering. So, is it possible to push data to nested folders? please can you explain with any dummy example
@HASecuritySolutions
@HASecuritySolutions 2 года назад
@@pulkitdikshit6148 do you mean nested objects? I'm uncertain what you mean by nested folders
@nikhili9559
@nikhili9559 2 года назад
great stuff :)
@HASecuritySolutions
@HASecuritySolutions 2 года назад
Thank you
Далее
SPONGEBOB POWER-UPS IN BRAWL STARS!!!
08:35
Просмотров 20 млн
Секрет фокусника! #shorts
00:15
Просмотров 29 млн
Elasticsearch (Part 1): Indexing and Querying
31:16
Просмотров 132 тыс.
Python Logging - Tutorial
15:02
Просмотров 158 тыс.
How to Build Chatbot with Python & Rasa
2:28:05
Просмотров 49 тыс.
Enrich your Data in Elasticsearch
14:43
Просмотров 2,8 тыс.