Solving a permission problem when setting up Hadoop to support multiple users

And now for something completely different…

If you are setting up a Hadoop cluster for multiple users and run into the following error:

...
AccessControlException: Permission denied: user=John, access=EXECUTE
...

while trying to run MapReduce jobs as users that aren’t the owner of the Hadoop files and folders in the system, then try the following:

hadoop fs –chmod -R 1777 /tmp

which fixed the issue for me.

And yes, the actual staging folder for each user is nested way, way deep inside the tmp folder on the HDFS, but for some reason it isn’t enough to change the permission of the staging folder, you need to change the permission of the tmp folder and all of the files and folders inside it as well.

Advertisements