1. Get the task attempt ID for the currently running Hadoop task stackoverflow.comThe Task Side-Effect Files section of the Hadoop tutorial mentions using the "attemptid" of the task as a unique name. How do I get this attempt ID in ... |
2. Can I set task memory limit higher than 2GB stackoverflow.comHadoop map-reduce configuration provides the mapred.task.limit.maxvmem and mapred.task.default.maxvmem. According to the documentation both of these are values of type long that is anumber, in bytes, that represents the default/upper ... |
3. Hadoop task schedulers: Capacity vs Fair sharing or something else? stackoverflow.comBackgroundMy employer is progressively shifting our resource intensive ETL and backend processing logic from MySQL to Hadoop ( dfs & hive ). At the moment everything is still somewhat small ... |
4. Hadoop Pipes: how to pass large data records to map/reduce tasks stackoverflow.comI'm trying to use |
5. What types of tasks / applications can use Apache Hadoop for (MapReduce functions) stackoverflow.comI don't understand what types of apps can be used with Hadoop. Does each task have to be tailored for hadoop/mapreduce. For example, can you just associate any long ... |
6. Hadoop, running tasks stackoverflow.comHow do I programmatically add tasks to hadoop and run in my Java application? Any ideas? Thanks. |
7. number of reducers for 1 task in MapReduce stackoverflow.comIn a typical MapReduce setup(like Hadoop), how many reducer is used for 1 task, for example, counting words? My understanding of that MapReduce from Google means only 1 reducer is involved. ... |
8. Hadoop Fair Scheduler not assigning tasks to some nodes stackoverflow.comI'm trying to run the Fair Scheduler, but it's not assigning Map tasks to some nodes with only one job running. My understanding is that the Fair Scheduler will use ... |
9. General Method for Determining Hadoop Conf Settings on a Single Node Cluster stackoverflow.comI am wondering how best to determine the appropriate numbers of map and reduce tasks and the corresponding maximum size of the JVM heap? For those new to Hadoop these ... |
10. Getting all TaskAttempts of a Task from Hadoop API stackoverflow.comI would like to get information on all TaskAttempts of a Task of a Job on Hadoop. org.apache.hadoop.mapred.TaskReport gives information on running Attempts and successful Attempts, but I would like to also ... |
11. How to expose the task tracker/Job tracker webinterface to the public in hadoop? stackoverflow.comI'm trying to monitor different cluster nodes, but everytime I have to ssh -X to the node and start the browser to take a look at the status information. Is there anyway ... |
12. Why does a number of completed tasks in Mapreduce decrease? stackoverflow.comWhen running hadoop jobs, I noticed that sometimes the number of completed tasks decreases and number of canceled tasks increases. How is this possible? Why does this happen? |
13. How does Hadoop transfer user-defined parameters to tasks? stackoverflow.comIn Hadoop, how does it transfer user's configuration parameter to the task? For example, conf.get("myparameter").
Is it through Serializable? and ... |
14. How to tell Hadoop to not delete temporary directory from HDFS when task is killed? stackoverflow.comBy default, hadoop map tasks write processed records to files in temporary directory at |