public class DefaultResourceIsolationManager extends Object implements ResourceIsolationInterface
This is the default class to manage worker processes, including launching, killing, profiling and etc.
| Modifier and Type | Field and Description |
|---|---|
protected Map<String,Object> |
conf |
protected boolean |
runAsUser |
| Constructor and Description |
|---|
DefaultResourceIsolationManager() |
| Modifier and Type | Method and Description |
|---|---|
boolean |
areAllProcessesDead(String user,
String workerId)
Check if all the processes are dead.
|
void |
cleanup(String user,
String workerId,
int port)
This function will be called when the worker needs to shutdown.
|
void |
forceKill(String user,
String workerId)
Kill the given worker forcefully.
|
protected Set<Long> |
getAllPids(String workerId)
Get all the pids that are a part of the container.
|
long |
getMemoryUsage(String user,
String workerId,
int port)
Get the current memory usage of the a given worker.
|
long |
getSystemFreeMemoryMb()
Get the amount of free memory in MB.
|
boolean |
isResourceManaged()
This class doesn’t really manage resources.
|
void |
kill(String user,
String workerId)
Kill the given worker.
|
void |
launchWorkerProcess(String user,
String topologyId,
Map<String,Object> topoConf,
int port,
String workerId,
List<String> command,
Map<String,String> env,
String logPrefix,
ExitCodeCallback processExitCallback,
File targetDir)
After reserving resources for the worker (i.e.
|
void |
prepare(Map<String,Object> conf)
Called when starting up.
|
void |
reserveResourcesForWorker(String workerId,
Integer workerMemory,
Integer workerCpu,
String numaId)
This function should be used prior to starting the worker to reserve resources for the worker.
|
boolean |
runProfilingCommand(String user,
String workerId,
List<String> command,
Map<String,String> env,
String logPrefix,
File targetDir)
Run profiling command.
|
public void prepare(Map<String,Object> conf) throws IOException
ResourceIsolationInterfaceCalled when starting up.
prepare in interface ResourceIsolationInterfaceconf - the cluster configIOException - on any error.public void reserveResourcesForWorker(String workerId, Integer workerMemory, Integer workerCpu, String numaId)
ResourceIsolationInterfaceThis function should be used prior to starting the worker to reserve resources for the worker.
reserveResourcesForWorker in interface ResourceIsolationInterfaceworkerId - worker id of the worker to startworkerMemory - the amount of memory for the worker or null if not enforcedworkerCpu - the amount of cpu for the worker or null if not enforcednumaId - NUMA zone if applicable the worker should be bound topublic void cleanup(String user, String workerId, int port) throws IOException
ResourceIsolationInterfaceThis function will be called when the worker needs to shutdown. This function should include logic to clean up after a worker is shutdown.
cleanup in interface ResourceIsolationInterfaceuser - the user of the workerworkerId - worker id to shutdown and clean up afterport - the port of the workerIOExceptionpublic void launchWorkerProcess(String user, String topologyId, Map<String,Object> topoConf, int port, String workerId, List<String> command, Map<String,String> env, String logPrefix, ExitCodeCallback processExitCallback, File targetDir) throws IOException
ResourceIsolationInterfaceAfter reserving resources for the worker (i.e. calling reserveResourcesForWorker), this function can be used to launch worker process.
launchWorkerProcess in interface ResourceIsolationInterfaceuser - the user who runs the command astopologyId - the Id of the topologytopoConf - the topology configurationport - the port where the worker is onworkerId - the Id of the workercommand - the command to runenv - the environment to run the commandlogPrefix - the prefix to include in the logsprocessExitCallback - a callback for when the process exitstargetDir - the working directory to run the command inIOException - on I/O exceptionpublic long getMemoryUsage(String user, String workerId, int port) throws IOException
ResourceIsolationInterfaceGet the current memory usage of the a given worker.
getMemoryUsage in interface ResourceIsolationInterfaceuser - the user that the worker is running asworkerId - the id of the workerport - the port of the workerIOException - on I/O exceptionpublic long getSystemFreeMemoryMb()
throws IOException
ResourceIsolationInterfaceGet the amount of free memory in MB. This might not be the entire box, it might be within a parent resource group.
getSystemFreeMemoryMb in interface ResourceIsolationInterfaceIOException - on I/O exceptionpublic void kill(String user, String workerId) throws IOException
ResourceIsolationInterfaceKill the given worker.
kill in interface ResourceIsolationInterfaceuser - the user that the worker is running asworkerId - the id of the worker to killIOException - on I/O exceptionpublic void forceKill(String user, String workerId) throws IOException
ResourceIsolationInterfaceKill the given worker forcefully.
forceKill in interface ResourceIsolationInterfaceuser - the user that the worker is running asworkerId - the id of the worker to killIOException - on I/O exceptionprotected Set<Long> getAllPids(String workerId) throws IOException
Get all the pids that are a part of the container.
IOExceptionpublic boolean areAllProcessesDead(String user, String workerId) throws IOException
ResourceIsolationInterfaceCheck if all the processes are dead.
areAllProcessesDead in interface ResourceIsolationInterfaceuser - the user that the processes are running asworkerId - the id of the worker to killIOException - on I/O exceptionpublic boolean runProfilingCommand(String user, String workerId, List<String> command, Map<String,String> env, String logPrefix, File targetDir) throws IOException, InterruptedException
ResourceIsolationInterfaceRun profiling command.
runProfilingCommand in interface ResourceIsolationInterfaceuser - the user that the worker is running asworkerId - the id of the workercommand - the command to runenv - the environment to run the commandlogPrefix - the prefix to include in the logstargetDir - the working directory to run the command inIOException - on I/O exceptionInterruptedException - if interruptedpublic boolean isResourceManaged()
This class doesn’t really manage resources.
isResourceManaged in interface ResourceIsolationInterfaceCopyright © 2021 The Apache Software Foundation. All rights reserved.