0

Security Considerations in Blue-Green Deployments

-

tl,dr; Blue-Green deployments for critical uptime applications is a strong deployment strategy but if a deployment fixes critical security issues be sure that the definition of “deployment complete” is decommissioning of the “blue” environment and not just deployment of “green” successfully.

Organizations have gotten used to following Continuous Integration/Continuous Deployment (CI/CD) for software releases. The use of cloud solutions such as AWS Code* utilities, Azure DevOps or Google Cloud Source repositories enables enterprises to quickly and securely accomplish CI/CD for software release cycles. Software upgrades undergo the same CI/CD tooling and the dev teams need to choose how to do the upgrades.

There are a few different ways development teams upgrade their applications – full cut-over (same infrastructure, new codebase deployed directly and migrated in one go), rolling deployments (same infra or new infra gradually upgrading all instances), immutable (brand new infra and code for each deployment and migrated in one go), blue-green deployment (same infra, simultaneously deployed in prod, gradually phasing out old instances upon successful tests from a section of traffic). The Blue-green deployment strategy has, therefore, become quite popular for modern software deployment.

What is a Blue-Green deployment?
When you release an application via Blue-Green deployment strategy you gradually shift traffic as tests succeed and your observability (Cloudwatch alarms, etc.) does not indicate any problems. You can do that via Containers (ECS/EKS/AKS/GKE) or AWS Lambda/Azure Functions/Google Cloud Functions and traffic shifting can be done with the help of DNS solutions (Route53/Azure DNS/Google Cloud DNS), Load balancing solutions (AWS Elastic Load Balancer/Azure Load Balancer/Cloud Load Balancing). Simplistically, take your current (blue) deployment and create a full stack (green) and use either DNS or load balancers to slice out a traffic section and test the “green” stack. This is all happening in production, by the way. Once everything looks good, direct all the traffic to green and decommission the “blue”. This helps maintain operational resilience and, therefore, this is a popular deployment strategy. AWS has solid whitepaper I recommend to review to dive in from a solution architecture standpoint if you are interested.

Security Considerations

Some critical security issues (e.g., remote code execution via Log4j, remote code execution via struts, etc.) demand immediate fixes because of their severity. If your blue-green deployments are going to take days and your tests will run over a very long period (say days) then any security fixes you make also will get fixed after a successful “green” deployment and “blue decommission” only. If during that window or prior, an attacker managed to get a foothold into the impacted “blue” environment, then even decommissioning of the “blue” becomes critical to claim the issue is fully remediated. Typically, when incident responders and security operations professionals breathe a sigh of relief is when the fix is deployed. Typically, the software engineering teams consider fix as deployed is when the “green” is fully handling all traffic (and its unrelated to the decommissioning of the “blue” environment). In this case, the incident responders need to remember its not the deployment time when the risk is truly mitigated, its mitigated after completion of cut-over to green and decommissioning of the blue. There is a subtle, yet important, difference here – and it really comes down to the use of shared vocabulary. As long as security operations and software development teams both have this shared definition of what deployment means, there are no misunderstandings.

0

404 Errors – Do I need to know what I requested?

-

A very typical scenario is that by default the Tomcat Servers tend to have a 404 Error page that displays the name of the file that was requested and not found. It seems to me that though display of such pages might be considered as merely an informational item for the purpose of any security test…this definitely presents a risk.
E.g., take this scenario.
1. The attackers has a SQL injection vulnerability in an application
2. The App server and DB can reach each other
3. The DB cannot directly reach the attacker and his system (on any port outbound)
4. The app server issues 404 error messages with the name of the file being disclosed in the 404 error message (e.g., The requested resource indexblah.html was not found).
5. The attacker can see the responses to the injected SQL queries (i.e., the injection is not blind).

Assuming that the DB accesses have been tightly controlled and you can’t get much access to any tables except the current one. This can be exploited as follows:
Invoke a SQL query (in the injection string) on the DB to request a page from the app server based on the contents of the DB such as send me /blahusername, /blahpassword where /blah is a string the attacker’s put in to make sure that such a resource doesn’t actually exist on the app server and username and password are columns or DB names from the DB. These error messages will be reflected in the response to the SQL query to the attacker. This could create an interesting side-channel attack whereby even though the data from the DB doesn’t actually reach the attacker, it can be inferred from the 404 – error messages.


___________ ______________ __________
| Attacker| <===> | App server | <=====>| DB |
___________ ______________ __________
1 ----------> 2 -----------------> 3
5 <---------------- 4
6 -----------------> 7
10 <---------- 9 <----------------- 8
1. Attacker sends sql to make the db query the app server for a non-existent page
2. The app server sends this sql query to the DB
3. The DB receives this SQL query and acts on it
4. The HTTP query for a missing resource is sent to the app server
5. App server looks up the resource and can't find it
6. The App server responds with a 404 /blahusername not found
7. The response recd is put in the SQL query response
8. The SQL query response is sent to the app server
9. The App server received the SQL query response (404 /blahusername not found as a line in there)
10. The attacker receives the 404 response with the data from the username in the 404 error message

An interesting attack vector to say the least!

1

ToorConX in San Diego

-

I recently came back from the ToorConX in San Diego, CA. It was a great conference with some really cool talks. Especially RFD (Remote file download using blind sql injection), reversing malware using browser hooking, cracking crypt() hashes using Ps3, grey box peach fuzz xml generation tool (nunchaku), voip eavesdropping, bypassing browser memory protection.
These were some really cool talks but I still don’t have access to any presentation slides yet for the con. May be those will be posted some time.

1

Cisco Router Security

-

Long time since I posted anything here …. but it’s just been those times been busy as a bee. So securing Cisco routers is a big deal especially since the routers (especially the edge routers) can be critical to any organizations infrastructure. I am not a Cisco guru but am only a student. However, I thought I should create a list that could help me perform security reviews of routers.
Security of routers is important as attackers could add static routes, advertise bad BGP neighbours on edge routers, create inbound tunnel into the intranets and such. Therefore, it’s imperative that adequate efforts be put in to secure Cisco routers.

I thought I’ll put in my first attempt at creating a small checklist:

  1. Use SSH for non-console access (“line vty” command should not have telnet in it)
  2. Use class 5 passwords, do *not* use class 7 passwords as they’re easily reversed (“enable secret”) alongwith the use of strong passez
  3. Limit virtual terminal access by using an ACL
       access-list 100 permit 10.10.10.10 log
       access-list 100 permit 10.10.10.11 log
       access-list deny any log
       line vty 0 4
        access-class 100 in
  4. Disable Proxy ARP on each interface (“no ip proxy arp”)
  5. Disable CDP as it can be used for information disclosures (“no cdp run”)
  6. Use AAA (TACACS+ or RADIUS) (“aaa new-model”, “aaa authentication”, etc.)
  7. Use “access-list ACL_NAME deny ip any any log” at the end of each ACL
  8. Disable http server (“no ip http server”)
  9. Keep the IOS versions updated
  10. Set centralized logging using a syslog (“logging internal_ip_address”)
  11. Configure NTP to keep the time synchronization (“ntp server 129.6.15.28”)
  12. Disable TCP and UDP small services e.g., echo, chargen, discard, etc. (“no service tcp-small-servers” and “no service udp-small-servers”)
  13. Put RFC 1918 (ingress filtering) protections using ACLs
       access-list 100 deny ip 127.0.0.0 0.255.255.255 any log
       access-list 100 deny ip 10.0.0.0 0.255.255.255 any log
       access-list 100 deny ip 192.168.0.0 0.0.255.255 any log
       access-list 100 deny ip 172.16.0.0 0.15.255.255 any log
  14. Put some more filtering for common IPs
       access-list 100 deny ip 169.254.0.0 0.0.255.255 any log
  15. Use SNMPv3 with ACLs if you must (“snmp-server v3 auth priv”)
  16. Use SSHv2 (“ip ssh version 2”)
  17. Try to use EIGRP instead of RIP/OSPF (“ip authentication mode eigrp N md5”)
  18. Use MD5 authentication for RIP/OSPF if you must use these protocols (RIPv2/OSPF)
    (“ip rip authentication mode md5”)
  19. For edge routers using BGP authentication (if possible)
       router bgp 10
        neighbor 10.10.10.10 password Cr4zY$%^
  20. Configure BGP route flap dampening that prevents BGP oscillations (“bgp dampening”)
  21. Use warning banners that could be used for legal purposes for prosecuting hackers
0

Compiling wepattack on backtrack4

-

I encountered various errors when compiling wepattack. This download does not come with a makefile that is compatible with the ubuntu distro that backtrack uses. First of all make sure that the wlan directory that you get when untarring the .tar.gz archive has execute permissions set to it.

$ cd WepAttack-0.1.3/src
$ chmod +x wlan

Once this is done “permission denied” errors should go.

/Desktop/WepAttack-0.1.3/src$ make
gcc -fno-for-scope -c -D__LINUX_WLAN__ -D__I386__ -o wepattack.o wepattack.c
cc1: warning: command line option "-fno-for-scope" is valid for C++/ObjC++ but not for C
wepattack.c: In function ‘loop_packets’:
wepattack.c:141: warning: incompatible implicit declaration of built-in function ‘strlen’
wepattack.c:146: warning: incompatible implicit declaration of built-in function ‘strlen’
wepattack.c:151: warning: incompatible implicit declaration of built-in function ‘strlen’
wepattack.c:156: warning: incompatible implicit declaration of built-in function ‘strlen’
wepattack.c: In function ‘clean_up’:
wepattack.c:184: warning: format ‘%d’ expects type ‘int’, but argument 3 has type ‘long int’
wepattack.c: In function ‘main’:
wepattack.c:309: warning: format ‘%d’ expects type ‘int’, but argument 2 has type ‘long int’
gcc -fno-for-scope -c -D__LINUX_WLAN__ -D__I386__ -o rc4.o rc4.c
cc1: warning: command line option "-fno-for-scope" is valid for C++/ObjC++ but not for C
gcc -fno-for-scope -c -D__LINUX_WLAN__ -D__I386__ -o wepfilter.o wepfilter.c
cc1: warning: command line option "-fno-for-scope" is valid for C++/ObjC++ but not for C
gcc -fno-for-scope -c -D__LINUX_WLAN__ -D__I386__ -o log.o log.c
cc1: warning: command line option "-fno-for-scope" is valid for C++/ObjC++ but not for C
gcc -fno-for-scope -c -D__LINUX_WLAN__ -D__I386__ -o modes.o modes.c
cc1: warning: command line option "-fno-for-scope" is valid for C++/ObjC++ but not for C
modes.c:25:30: error: wlan/wlan_compat.h: Permission denied
modes.c:26:28: error: wlan/p80211hdr.h: Permission denied
modes.c: In function ‘generate_rc4_key’:
modes.c:51: warning: incompatible implicit declaration of built-in function ‘memcpy’
modes.c: In function ‘process_rc4_key’:
modes.c:68: warning: incompatible implicit declaration of built-in function ‘memcpy’
modes.c: In function ‘mode_keygen’:
modes.c:125: warning: incompatible implicit declaration of built-in function ‘memcpy’
modes.c:127: warning: incompatible implicit declaration of built-in function ‘strcpy’
modes.c: In function ‘mode_wep’:
modes.c:145: warning: incompatible implicit declaration of built-in function ‘memcpy’
make: *** [modes.o] Error 1

The following patch file will take care of most errors and you should be able to get Wepattack compiled properly:

diff -aur WepAttack-0.1.3/src/Makefile WepAttack-patched/src/Makefile
--- WepAttack-0.1.3/src/Makefile 2002-10-23 09:11:36.000000000 -0400
+++ WepAttack-patched/src/Makefile 2010-09-26 04:54:20.000000000 -0400
@@ -6,23 +6,23 @@
LD=gcc
#
# CFLAGS
-CFLAGS=-fno-for-scope -c -D__LINUX_WLAN__ -D__I386__
+CFLAGS= -c -D__LINUX_WLAN__ -D__I386__
#
#
# LDFLAGS
-#LDFLAGS=
+LDFLAGS=-L../run
#
#
# Libraries to link against
-LIBS= -lpcap -lz -lcrypto
+LIBS= -lpcap -lz -lcrypto
#
#
# Install path for wepattack
INSTDIR=/usr/bin

+INCLUDEDIR=-Isrc/
wepattack: wepattack.o rc4.o wepfilter.o log.o modes.o misc.o verify.o keygen.o
- $(LD) $(LDFLAGS) -o $@ wepattack.o rc4.o wepfilter.o log.o\
- modes.o misc.o verify.o keygen.o $(LIBS)
+ $(LD) $(LDFLAGS) $(INCLUDEDIR) -o $@ wepattack.o rc4.o wepfilter.o log.o modes.o misc.o verify.o keygen.o $(LIBS)

wepattack.o: wepattack.c wepattack.h
$(CC) $(CFLAGS) -o $@ wepattack.c
@@ -46,7 +46,7 @@
$(CC) $(CFLAGS) -o $@ keygen.c

modes.o: modes.c modes.h
- $(CC) $(CFLAGS) -o $@ modes.c
+ $(CC) $(CFLAGS) $(INCLUDEDIR) -o $@ modes.c

misc.o: misc.c misc.h
$(CC) $(CFLAGS) -o $@ misc.c
diff -aur WepAttack-0.1.3/src/modes.c WepAttack-patched/src/modes.c
--- WepAttack-0.1.3/src/modes.c 2002-10-24 09:15:19.000000000 -0400
+++ WepAttack-patched/src/modes.c 2010-09-26 04:55:22.000000000 -0400
@@ -29,6 +29,7 @@
#include "wepattack.h"
#include "wepfilter.h"
#include "verify.h"
+#include "string.h"

static rc4_key gen_key;
static unsigned char decrypted_stream[2400];
Only in WepAttack-patched/src: wepattack
diff -aur WepAttack-0.1.3/src/wepattack.c WepAttack-patched/src/wepattack.c
--- WepAttack-0.1.3/src/wepattack.c 2002-10-24 09:14:29.000000000 -0400
+++ WepAttack-patched/src/wepattack.c 2010-09-26 04:41:18.000000000 -0400
@@ -36,7 +36,7 @@
#include "config.h"
#include "modes.h"
#include "misc.h"
-
+#include

wlan_packet_list* current_packet;

@@ -181,7 +181,7 @@

// calculate elapsed time
duration = difftime_us(&t_val_start, &t_val_end);
- printf("\ntime: %f sec\twords: %d\n\n", duration, word_count);
+ printf("\ntime: %f sec\twords: %ld\n\n", duration, word_count);

// write ucracked packets to logfile
log_uncracked(list_packet_to_crack);
@@ -306,7 +306,7 @@

// print out each 10'000 key
if ((word_count % 10000) == 0)
- printf("key no. %d: %s\n", word_count, key);
+ printf("key no. %ld: %s\n", word_count, key);
word_count++;

// main loop to process key in modes on every packet

Copy the above patch in to a file called wepattack.patch. Copy wepattack.patch into the WepAttack-0.1.3 directory and patch it as follows:

$ patch -p1 <wepattack.patch
$ cd src
make
sudo make install

You should be able to get wepattack installed!

0

GooScan compilation errors

-

I was just browsing away when I stumbled upon Johnny Long’s GooScan. He says that this is a Linux only tool but it seems to compile (not without problems though) on cygwin.
I kept getting the following errors:


L:\tools\gooscan-v1.0.9>gcc gooscan.c
gooscan.c: In function `inet_send':
gooscan.c:575: error: `MSG_WAITALL' undeclared (first
use in this function)
gooscan.c:575: error: (Each undeclared identifier is
reported only once
gooscan.c:575: error: for each function it appears in.)

Then I read somewhere that MSG_WAITALL is not defined for Cygwin and that instead of that zero would work. There are many neater solutions to this…but I’m a hacker and I’ll do the stuff that’s easiest and hassle-free.
Some people say that the following will work:
#ifdef __CYGWIN__
#define MSG_WAITALL 0

So in order to compile this bad boy, you need to goto line 574 in your favorite editor.
It looks like this:
recv(sock, recvbuf, sizeof(recvbuf), MSG_WAITALL);

You need to make it look like this:
recv(sock, recvbuf, sizeof(recvbuf), 0);//MSG_WAITALL);

You are all set:
gcc gooscan.c -o gooscan.exe

Compilation works! But then I observed that the results were not coming well. However, if you run it through a local proxy such as burp it still works…I bet it has something to do with socket establishment and receiving and being incompatible with the MSG_WAITALL flag.
But as long as you can get the results … who cares? If someone figures out exactly how to make this work, please post it as a comment.

0

Nmap and DNS resolution Timeouts

-

I think Nmap is by far the best portscanner around if you want to do some serious port-scanning. Nmap performs a DNS resolution by default. This is good for obtaining the fully qualified domain names (FQDN), however, in some cases when you are scanning huge networks spanning several class Bs, it can have a significant effect on the duration of the scan.
Although using the -n parameter can completely stop nmap from performing any resolutions, but sometimes there’s that fine granularity that you need, i.e., you want to perform name resolutions but not if it exceeds a certain amount of time. I have to say that I wouldn’t have even craved for such an idiosyncratic feature, had it not been for nmap. Fyodor has been awesome enough to provide fine-grained control over port-scanning to your heart’s content.
So I opened up the nmap code, trying to figure out if I could fine tune that feature myself and I was not at all surprised that there were several comments in the code that would give you the impression that the authors of nmap have been considering this feature.
At this time it seems that the timeouts for the DNS servers are being read out of an arrayname:
static int read_timeouts[][] in nmap_dns.cc.
The way the code works is, this array has retransmission timeouts. Each row of this array represents what retransmission timeouts that nmap will follow depending on the number of DNS servers provided.

In nmap 4.76, therefore, if you specify one DNS server (or only one entry exists in /etc/resolv.conf) nmap will wait 4000ms, then another 4000ms followed by 5000ms before giving up. But if you do specify two DNS servers, then for the first DNS server the timeouts are 2500ms followed by 4000ms and then the same is tried for the 2nd entry in the DNS servers. Therefore, it seems that nmap will wait 13 seconds at max before giving up on the DNS resolution of a host. Imagine scanning a class B and having to wait 13 seconds for each of the hosts to resolve. It would be a significant overhead.

Of course, one can find other things to do if the IP address space is not DHCP, e.g., starting a separate list scan (-sL) and a portscan (with -n) simultaneously so that the DNS resolution timeouts do not result in a major impact as far as the portscanning itself is concerned.
There could be pros and cons to this as well which I may have failed to consider. But at this time it seems that it might be the most judicious approach.