As promised, this is my additional notes and review about my Keynote talk in 2019.HACK.LU (link). My keynote talk title is very long actually, but it explained the description of the whole slides clearly. What was presented is about TODAY's Linux post exploitation, process injection, fileless execution from infrastructures and components that has been supporting those activities, based on the real incidents handled within these past seven years of MalwareMustDie project, from the blue-teamer point of view. The goal is for forming a better defense on our Linux platform and some methods for IR in handling those incidents.
About 2019.HACK.LU
HACK.LU is a great conference, thank you for having me this year, I could interact with a lot of infosec community who I already know but haven't met them until now, and I could also get along with old friends in the community too. 2019.HACK.LU had a warm environment, that I can trust and I met with a lot of good people from various IT security background, from various countries. It has important atmosphere to support an open and productive discussion between great professionals in the event. And a nice comparison to make between what we have in Asia/Japan to Europe.
It's not a new thing, but I can't help to notice that several new acquaintances looks surprised meeting me at the first time :) It was like "who is this guy?" or "Really?", So, guys, this is who I am! Let's don't judge people by their appearance, free your minds..like a pointer, so you can use it in an unlimited possibilities!
As you all maybe know by now that I am working in the IR field on handling malware intrusion cases for supporting forensics in daily basis, I don't have much chances going out to international conference to hang out with others, and instead, recently I have returned to write on this blog again after three years full on doing assessments of our infrastructures and fixing several OPSEC aspect we had (along with daily work matters, and I work hard too). But as MMD, our research never stops, and one of research that we are doing has just been presented in the 2019.HACK.LU. You can see those on our tweets, our Linux malware repository, our quick analysis screenshot posts and other media.
About the talk & some extra reading takeaways
As the lead of MMD, from our side, officially we don't openly share any slides or video for this talk due to the security purpose. It's because, within these past three (more) years we have figured that more of the "dark-siders" are reading our security posts more than whitehats, and that's never good. So this time I don't want to give adversaries much luxury in learning from, and, accessing our research material(s) that easily, this is explaining the TLP we put on the slides, it's "TLP Amber for our ranks" as the sharing policy. But I think the conference attendees and other good people has already received access to the material too.
The presentation is full of contents, very thorough actually, but there was so limited time to explain each concept in details on the stage because I was focusing to cover them all within 45 minutes. So, after doing several one-on-one explanation sessions to the audiences afterward, I think many of you should go look deeper beforehand into several Linux function's source codes that may trigger a "mis-using possibility" that may cause any malicious scheme to run a code or program.
For those who have the interest, I am urging you to look at the methods I presented further (in order to learn or to mitigate them more in kill-chain basis), by reading more about the related references that is needed to better understand the concepts written in the slides and its possibility to other parts of code that hasn't be described too.
Therefore herewith I listed below some reference for the above purpose:
- Commands listed in slide between page 26 and page 38
- ELF execution references that I presented in R2CON2018
- Details listed in slide page 43, 93, and page 96
- GTFO_Bins in Linux & its privilege escalation
- ELF binary compilation using PIE
- SE Linux, what can it actually does & doesn't do.
- Linux ALSR methods & its affect to process execution (incl.: forking, threading, libs calling).
- Buffer exploitation (smashing stack) & modify ret addr for code execution.
- Linux shared objects, how their libraries can be abused
- ptrace() (how the gdb and dbx are using it)
- /proc/{pid}/maps, /proc/{pid}/mem
- memfd_create()
- shm_open()
- dlopen_mode()
- LD_PRELOAD
- (I may add some gems that I forgot)
You can use the examples of three cases in the slide to test and assess the understanding of your IR team in handling Linux intrusion, and maybe regenerate several cases I presented by your own too. It is always bringing much advantages to you for doing more practice, trust me, many practice makes perfect!
Several questions and answers received in 2019.HACK.LU
I received a lot of questions (and cool tributes after keynote talk too, thank you for those). For the questions I tried to list them all in a list as per below, these items may contain one of your questions too, so please read them all before you start to ask your question via the internet.
Q: What is the bottom line of the presentation?
A: The point of my presentation is for all of us to be aware more that Linux deserves better implementation in security than we are having now (seriously, we don't have much options in protecting Linux OS compare to what we have in Windows, just saying), and the kill-chain methods to prevent us from post exploitation in the Linux platform can be applied better after we understand and after we learn to make a breakdown process in steps and its intrusion assessment possibility in the platform. The talk I did was just the start point for this level of awareness.
Q: So the talk you did was like the PoC of the threat?
A: No, the presentation is explaining the the bad activities that have already happened in the internet as attempts to do part of post-exploitation components (mostly process injections) on incidents that has been reported to us. There are more of information that is not being applied "in the wild", but I am not going to those information in the any open forum. The information I presented is the things that all blue-teamers should know, since the incidents for those concepts are actually had happened or had been implemented in some adversaries frameworks.
Q: Are there other concepts in the Linux libraries that can be used for the code injection?
A: Yes there are. not necessarily execution from the vulnerability sector, Linux is also rich of libraries to be used for the internal and development purpose. For example, the libc's dlopen_mode for example, it is used mostly to load libraries between libraries, but most of the people tend to just using Linux then understanding its source. Due to its open source state, the malicious misusing of Linux libraries can be performed by the adversaries without having much difficulties if the skill-sets are there, and the worst part is the combination of several good libraries to pivot ALSR for example, which can be used as combination to perform any kind of undetected malicious intent. We should start to open our eyes for these.
Q: About codes you wrote in the presentation can they be used?
A: I don't write any usable codes in the presentation, those are codes for explaining the concept I presented, I tweaked much of them in some (confusing, hopefully) ways. But coders know what I meant and they are following the points very well judging the discussion we had after keynote, so I'd said good luck for the copy-pasta adversary who would use the codes I presented.
Q: Are you suggesting that we should do the hot forensics instead of the cold one in Linux?
A: The idea of the talk is not pushing you to do IR works "on-memory", but to make all of us aware of the steps taken in the process injection by break them down into steps. By understanding this we can understand what can be more improve to prevent the intrusion like this to happen in the future, by killing the chains, and so on. "On-memory" analysis is do-able, with a ton of limitation of course, so it is not the top priority as the solution, but it would be very nice if you can do it, the examples in some cases presented are explaining how. Nevertheless, cold forensics is a must do too, however please keep it in mind that dealing cold forensics, like in dealing with with XFS/ZFS file system, is not as easy as what you think it is, and many more obstacle in that. So the best solution is applying kill-chain methods by some scheme, implementing more strict ALSR, making users using SELinux, applying backup, make sure that the memory can be dumped (did you ever test it?). The analysis can be done perfectly if the sysadmin is resourceful enough.
Q: Are you trying to push this project into the next level?
A: Yes, but for strictly IR/CERT/CSIRT related folks only, in example, we made the plan to do the workshop or hackathon topic for this subject in the next IR conference, and over there the repository of this project that we privately maintained will be announced. We don't go public for that matter, I will strictly transfer the know how to IR folks for incident handling.
Q: How to make sure that our boxes is not infected by code injection?
A: The best practice is to audit your box frequently, and for that you need to really knowing what's running on your box. Make sure there is no obsolete services or weak authentication and so on. What I can see in MMD handled incidents are, mostly coming from CGI vulnerabilities, services and its authentication vulnerabilities or leaks of access, these are what causing the intrusion at most.
Once the adversary gets into your system, it will be harder to figure. If a process runs in a weird way, do not hesitate to assess them, when you think it is necessary, unplug the box from the network. Many cool sysadmins gave us access to the boxes after they secured their data beforehand and together we examined the suspected processes.
Q: Why are you using radare2? Why not using other tools? What is the advantages of using radare2 instead of others for this matter?
A: I use a lot of RE tools, don't get me wrong, but I am in love with radare as tool since 2007 (or maybe 2006.. from my FreeBSD ports). radare2 is free for the license and for the concept in using it, it is just right tooling framework for the UNIX people, and I am using r2 daily for a lot of UNIX platforms and RTOS on many architectures. It is maybe hard for the first timers, but it is a very nice tooling once you know it. radare2 is the only one RE tool that support forensics and having four de-compiling engines to the binaries from wide range of platform. Its license is making me easier to lecture about RE or forensics to others.
Q: Why are you working under FreeBSD in analyzing Linux?
A: Why not? All OS is good, my first desktop is Solaris Sparc (Ultra1), BSD is cool. OSX is awesome. Linux is good, it's very popular and used in many devices, which is good too, this is just a flip-side of that situation, that's it. Remember, do stuff that you think you're good at, that includes to use your own favorite OS. And radare2 can work perfect in any OS at any kind of CPU, so I really have no problem about OS.
Q: How to contact you or passing something to you for analysis?
A: For the contact. You can send DM to our account in twitter at @malwaremustd1e, try to explain first about yourself and the situation, we will try follow for the urgency with some advise or action. For the sample to analyze please use this URL to send your sample to us. We will try to take a look based on our priority.
Q: Do you have other more Linux security research to share in events?
A: Yes, I have one more open topics and two close/sensitive ones for Linux, feel free to contact if you want us to present on your events, we will pick a right subject. But I think I will not talk in conference about threat actor(s) anymore, but more into academia or educational sections, or to support RE, IR or DFIR works from now on.
Salutation and thank you
I firstly thank HackLU, CIRCL and LAC/LACERT for they are the one who are really paying the bills for this talk to happen. I can't thank enough to the friends in FIRST.org, it's a small world, and we really are doing wonderful things, and we'll see you over there soon.
Thank's to MMD core team and good people supported to the presentation materials, I can't mention them enough in here. Thank you radare2 for the great RE tool. To all friends in Luxembourg whom we finally met, it's a pleasure, and looking forward to see you again, for the old friends, thank you for your great support always, for bearing with me in the baby-foot game or power-point karaoke or several conversation difficulties, you guys rock!
Below is the kind words feedback received in twitter and I am planning to put it in a frame to hang on the wall to remember you guys always, thank you! :)
Please support our Linux security infosec sharing projects in MalwareMustDie, there are many more hot posts are coming to our blog. Please help so we can also "re-generate" our resources to next generation as per seen in the below diagram.
As we are also leaners for many aspects, please do not hesitate to teach us on how to secure our sharing further.
For all of the blue-teamers out there, you're not alone, what we're doing is important and no matter how hard it is (see the picture below) just take it easy! Please hang in there with faith.
Thank you very much for your reading time! Stay safe!
#MalwareMustDie!