Hadoop java.io.IOException: Mkdirs failed to create /some/path
Just ran into this problem running mahout from CDH4 in standalone mode in my MacBook Air.
The issue is that a /tmp/hadoop-xxx/xxx/LICENSE file and a /tmp/hadoop-xxx/xxx/license directory are being created on a case-insensitive file system when unjarring the mahout jobs.
I was able to workaround this by deleting META-INF/LICENSE from the jar file like this:
zip -d mahout-examples-0.6-cdh4.0.0-job.jar META-INF/LICENSE and then verified it with
jar tvf mahout-examples-0.6-cdh4.0.0-job.jar | grep -i license Solution 3
The problem is OSX specific it is due to the fact that by default the filesystem is set to case-insensitive on a Mac (case preserving but case insensitive, which to my opinion is very bad).
A hack to circumvent this is to create a .dmg disk image with disk utility which is case sensitive and mount this image where you need it (i.e. hadoop.tmp.dir or /tmp) with the following command (as a superuser):
sudo hdiutil attach -mountpoint /tmp .dmg Solution 4
I ran into this issues several times in the past, I believe it is a Mac specific issue. Since I use Maven to build my project, I was able to get around it by adding a line in my Maven pom.xml like this:
org.apache.maven.plugins maven-shade-plugin 2.0 package shade Solution 5
In my case below lines of code in pom.xml in Maven project worked on Mac.
org.apache.maven.plugins maven-shade-plugin 2.0 true package shade *:* META-INF/*.SF META-INF/*.DSA META-INF/*.RSA META-INF/LICENSE* license/* Saved searches
Use saved searches to filter your results more quickly
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
java.io.IOException: Failed to mkdirs #11798
java.io.IOException: Failed to mkdirs #11798
Comments
failed log:
Building remotely on win201-528f87 (win2012-20180911) in workspace D:\j\w\perf_throughp—03c540a2@94
[WS-CLEANUP] Deleting project workspace.
java.io.IOException: Failed to mkdirs: D:\j\w\perf_throughp—03c540a2@94
at hudson.FilePath.mkdirs(FilePath.java:1170)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1200)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at com.cloudbees.plugins.flow.FlowRun.run(FlowRun.java:153)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
The text was updated successfully, but these errors were encountered:
@AaronRobinsonMSFT Jenkins issues will be with us as long as we’re on it, but that will likely change in the near future when we switch to Azure DevOps. We could see similar issues there since any system is going to have a certain spurious failure rate, but hopefully it’s better.
@RussKeldorph Sorry, you might be misunderstanding my question. Is there anything we can do if this happens in Jenkins other than restart the CI — i..e should this issue be closed because it isn’t something we can control?
@AaronRobinsonMSFT The individual job is not recoverable AFAIK, if that’s what you’re asking. Yes, you should just re-run the job and hope for the best if you need the results.
In theory the issue is not useless because there is absolutely «something we can do about it,» which would be to prevent it from happening by investigating what went wrong and preventing it from happening in the future. We aren’t going to do that, though, so closing it «won’t fix» is probably the best thing to do for now.
@RussKeldorph Thanks for the clarification. 👍
Footer
You can’t perform that action at this time.