Class BasicContainer
java.lang.Object
org.apache.storm.daemon.supervisor.Container
org.apache.storm.daemon.supervisor.BasicContainer
- All Implemented Interfaces:
Killable
A container that runs processes on the local box.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.storm.daemon.supervisor.Container
Container.ContainerType -
Field Summary
FieldsModifier and TypeFieldDescriptionprotected booleanprotected final doubleprotected final longprotected final LocalStateprotected final longprotected final longprotected final longprotected longprotected longprotected final Stringprotected final StringFields inherited from class org.apache.storm.daemon.supervisor.Container
assignment, conf, containerMemoryTracker, ops, port, resourceIsolationManager, runAsUser, supervisorId, supervisorPort, symlinksDisabled, topoConf, topologyId, type, workerId -
Constructor Summary
ConstructorsConstructorDescriptionBasicContainer(Container.ContainerType type, Map<String, Object> conf, String supervisorId, int supervisorPort, int port, LocalAssignment assignment, ResourceIsolationInterface resourceIsolationManager, LocalState localState, String workerId, StormMetricsRegistry metricsRegistry, ContainerMemoryTracker containerMemoryTracker) Create a new BasicContainer. -
Method Summary
Modifier and TypeMethodDescriptionvoidClean up the container partly preparing for restart.protected voidCreate a new worker ID for this process and store in in this object and in the local state.booleanReturn true if the main process exited, else false.frameworkClasspath(SimpleVersion topoVersion) getDependencyLocationsFor(Map<String, Object> conf, String topologyId, AdvancedFSOps ops, String stormRoot) longGet the current memory reservation of this container.longGet the current memory usage of this container.static StringgetStormVersionFor(Map<String, Object> conf, String topologyId, AdvancedFSOps ops, String stormRoot) protected StringgetWildcardDir(File dir) Returns a path with a wildcard as the final element, so that the JVM will expand that to all JARs in the directory.protected StringgetWorkerClassPath(String stormJar, List<String> dependencyLocations, SimpleVersion topoVersion) Compute the classpath for the worker process.protected StringgetWorkerLogWriter(SimpleVersion topoVersion) protected StringgetWorkerMain(SimpleVersion topoVersion) booleanisMemoryLimitViolated(LocalAssignment withUpdatedLimits) Check if the container is over its memory limit AND needs to be killed.protected Stringprotected StringjavaLibraryPath(String stormRoot, Map<String, Object> conf) Compute the java.library.path that should be used for the worker.voidlaunch()Launch the process for the first time.voidrelaunch()Restart the processes in this container.booleanrunProfiling(ProfileRequest request, boolean stop) Run a profiling request.substituteChildopts(Object value) substituteChildopts(Object value, int memOnheap, int memOffHeap) Methods inherited from class org.apache.storm.daemon.supervisor.Container
areAllProcessesDead, cleanUp, createArtifactsLink, createBlobstoreLinks, deleteSavedWorkerUser, forceKill, getTotalTopologyMemoryReserved, getTotalTopologyMemoryUsed, getTotalWorkersForThisTopology, getWorkerId, getWorkerUser, kill, readHeartbeat, readTopoConf, saveWorkerUser, setup, toString, updateMemoryAccounting, writeLogMetadata
-
Field Details
-
localState
-
profileCmd
-
stormHome
-
hardMemoryLimitMultiplier
protected final double hardMemoryLimitMultiplier -
hardMemoryLimitOver
protected final long hardMemoryLimitOver -
lowMemoryThresholdMb
protected final long lowMemoryThresholdMb -
mediumMemoryThresholdMb
protected final long mediumMemoryThresholdMb -
mediumMemoryGracePeriodMs
protected final long mediumMemoryGracePeriodMs -
exitedEarly
protected volatile boolean exitedEarly -
memoryLimitMb
protected volatile long memoryLimitMb -
memoryLimitExceededStart
protected volatile long memoryLimitExceededStart
-
-
Constructor Details
-
BasicContainer
public BasicContainer(Container.ContainerType type, Map<String, Object> conf, String supervisorId, int supervisorPort, int port, LocalAssignment assignment, ResourceIsolationInterface resourceIsolationManager, LocalState localState, String workerId, StormMetricsRegistry metricsRegistry, ContainerMemoryTracker containerMemoryTracker) throws IOExceptionCreate a new BasicContainer.- Parameters:
type- the type of container being made.conf- the supervisor configsupervisorId- the ID of the supervisor this is a part of.supervisorPort- the thrift server port of the supervisor this is a part of.port- the port the container is on. Should be <= 0 if only a partial recoveryassignment- the assignment for this container. Should be null if only a partial recovery.resourceIsolationManager- used to isolate resources for a container can be null if no isolation is used.localState- the local state of the supervisor. May be null if partial recoveryworkerId- the id of the worker to use. Must not be null if doing a partial recovery.metricsRegistry- The metrics registry.containerMemoryTracker- The shared memory tracker for the supervisor's containers- Throws:
IOException
-
-
Method Details
-
getDependencyLocationsFor
public static List<String> getDependencyLocationsFor(Map<String, Object> conf, String topologyId, AdvancedFSOps ops, String stormRoot) throws IOException- Throws:
IOException
-
getStormVersionFor
public static String getStormVersionFor(Map<String, Object> conf, String topologyId, AdvancedFSOps ops, String stormRoot) throws IOException- Throws:
IOException
-
createNewWorkerId
protected void createNewWorkerId()Create a new worker ID for this process and store in in this object and in the local state. Never call this if a worker is currently up and running. We will lose track of the process. -
cleanUpForRestart
Description copied from class:ContainerClean up the container partly preparing for restart. By default delete all of the temp directories we are going to get a new worker_id anyways. POST CONDITION: the workerId will be set to null- Overrides:
cleanUpForRestartin classContainer- Throws:
IOException- on any error
-
relaunch
Description copied from class:ContainerRestart the processes in this container. PREREQUISITE: cleanUpForRestart has run and passed- Specified by:
relaunchin classContainer- Throws:
IOException- on any error
-
didMainProcessExit
public boolean didMainProcessExit()Description copied from class:ContainerReturn true if the main process exited, else false. This is just best effort return false if unknown.- Specified by:
didMainProcessExitin classContainer
-
runProfiling
public boolean runProfiling(ProfileRequest request, boolean stop) throws IOException, InterruptedException Description copied from class:ContainerRun a profiling request.- Specified by:
runProfilingin classContainer- Parameters:
request- the request to runstop- is this a stop request?- Returns:
- true if it succeeded, else false
- Throws:
IOException- on any errorInterruptedException- if running the command is interrupted.
-
javaLibraryPath
Compute the java.library.path that should be used for the worker. This helps it to load JNI libraries that are packaged in the uber jar.- Parameters:
stormRoot- the root directory of the worker processconf- the config for the supervisor.- Returns:
- the java.library.path/LD_LIBRARY_PATH to use so native libraries load correctly.
-
getWildcardDir
Returns a path with a wildcard as the final element, so that the JVM will expand that to all JARs in the directory.- Parameters:
dir- the directory to which a wildcard will be appended- Returns:
- the path with wildcard ("*") suffix
-
frameworkClasspath
-
getWorkerMain
-
getWorkerLogWriter
-
getWorkerClassPath
protected String getWorkerClassPath(String stormJar, List<String> dependencyLocations, SimpleVersion topoVersion) Compute the classpath for the worker process.- Parameters:
stormJar- the topology jardependencyLocations- any dependencies from the topologytopoVersion- the version of the storm framework to use- Returns:
- the full classpath
-
substituteChildopts
-
substituteChildopts
-
javaCmd
-
isMemoryLimitViolated
Description copied from class:ContainerCheck if the container is over its memory limit AND needs to be killed. This does not necessarily mean that it just went over the limit.- Overrides:
isMemoryLimitViolatedin classContainer- Throws:
IOException- on any error
-
getMemoryUsageMb
public long getMemoryUsageMb()Description copied from class:ContainerGet the current memory usage of this container.- Overrides:
getMemoryUsageMbin classContainer
-
getMemoryReservationMb
public long getMemoryReservationMb()Description copied from class:ContainerGet the current memory reservation of this container.- Overrides:
getMemoryReservationMbin classContainer
-
launch
Description copied from class:ContainerLaunch the process for the first time. PREREQUISITE: setup has run and passed- Specified by:
launchin classContainer- Throws:
IOException- on any error
-