Quantcast
Channel: Hortonworks » All Replies
Viewing all articles
Browse latest Browse all 3435

Reply To: HDFS Transparent Data Encryption

$
0
0

Okay unfortunately no one has replied to this so I went ahead and tried something to test my setup.

Sharing the details below to save someone’s time :-)

Implemented TDE on single node HDP sandbox(hdp2.2).

Details – Installed and configured KMS, Created 2 encrypted zones on hdfs

1. /Secureweblogs with EZ key key1
2. /Huezone with EZ key hueKey

Output:

[hdfs@sandbox conf]$ hdfs crypto -listZones
/secureweblogs key1
/Huezone hueKey

As the ‘hive’ user, you can transparently write data to that directory.

Output :
[hive@sandbox ~]# hdfs dfs -copyFromLocal web.log /secureweblogs
[hive@sandbox ~]# hdfs dfs -ls /secureweblogs
Found 1 items
-rw-r–r–   1 hive hive       1310 2015-01-11 23:28 /secureweblogs/web.log
 
As the ‘hive’ user, you can transparently read data from that directory, and verify that the exact file that was loaded into HDFS is readable in its unencrypted form.

[hive@sandbox ~]# hdfs dfs -copyToLocal /secureweblogs/web.log read.log
[hive@sandbox ~]# diff web.log read.log
[hive@sandbox conf]$ hadoop fs -cat /secureweblogs/web.log
this is web log
[hive@sandbox conf]$

Other users will not be able to write data or read from the encrypted zone:

[root@sandbox conf]# hadoop fs -cat /secureweblogs/web.log
cat: Permission denied: user=root, access=EXECUTE, inode=”/secureweblogs”:hive:hive:drwxr-x—

hdfs superuser has access to raw namespace however he can only see encrypted contents not the actual data.

[hdfs@sandbox conf]$ whoami
hdfs
[hdfs@sandbox conf]$ hadoop fs -cat /.reserved/raw/secureweblogs/web.log
▒5 ▒0▒7s▒9▒]i▒▒▒
[hdfs@sandbox conf]$

Summary – you need to have strong ACL configured for this. we know that hdfs is hadoop superuser so it can access the encrypted contents but not the actual data!


Viewing all articles
Browse latest Browse all 3435

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>