Package org.apache.storm.daemon.nimbus
Class Nimbus
java.lang.Object
org.apache.storm.daemon.nimbus.Nimbus
- All Implemented Interfaces:
DaemonCommon,Shutdownable,Nimbus.Iface
-
Nested Class Summary
Nested Classes -
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final SimpleVersionstatic final Subjectstatic final List<org.apache.storm.shade.org.apache.zookeeper.data.ACL> -
Constructor Summary
ConstructorsConstructorDescriptionNimbus(Map<String, Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, TopoCache topoCache, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry) Nimbus(Map<String, Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry) -
Method Summary
Modifier and TypeMethodDescriptionvoidvoidaddSupervisor(Supervisor supervisor) Used for local cluster.booleanawaitLeadership(long timeout, TimeUnit timeUnit) beginBlobDownload(String key) beginCreateBlob(String key, SettableBlobMeta meta) beginUpdateBlob(String key) voidcancelBlobUpload(String session) voidvoidcheckAuthorization(String topoName, Map<String, Object> topoConf, String operation, ReqContext context) static voidcleanInbox(String dirLoc, int seconds) Deletes jar files in dirLoc older than seconds.voidvoiddeactivate(String topoName) voidEnable/disable logging the tuples generated in topology via an internal EventLogger bolt.voiddeleteBlob(String key) voidCleanup topologies and Jars.downloadBlobChunk(String session) downloadChunk(String id) filterAuthorized(String operation, Collection<String> topoIds) voidfinishBlobUpload(String session) voidfinishFileUpload(String location) voidforceDeleteTopoDistDir(String topoId) getBlobMeta(String key) intgetBlobReplication(String key) getComponentPageInfo(String topoId, String componentId, String window, boolean includeSys) getComponentPendingProfileActions(String id, String componentId, ProfileAction action) getLogConfig(String topoId) static List<org.apache.storm.shade.org.apache.zookeeper.data.ACL>getNimbusAcls(Map<String, Object> conf) getOwnerResourceSummaries(String owner) getSupervisorAssignments(String nodeId) Get assigned assignments for a specific supervisorgetSupervisorPageInfo(String superId, String host, boolean includeSys) getTopology(String id) Returns the compiled topology that contains ackers and metrics consumsers.getTopologyHistory(String user) getTopologyInfoByName(String name) getTopologyInfoByNameWithOpts(String name, GetInfoOptions options) getTopologyInfoWithOpts(String topoId, GetInfoOptions options) getTopologyPageInfo(String topoId, String window, boolean includeSys) Returns the user specified topology as submitted originally.static intgetVersionForKey(String key, NimbusInfo nimbusInfo, org.apache.storm.shade.org.apache.curator.framework.CuratorFramework zkClient) booleanisRemoteBlobExists(String blobKey) Decide if the blob is removed from cluster.booleanisTopologyNameAllowed(String name) booleanvoidkillTopology(String name) voidkillTopologyWithOpts(String topoName, KillOptions options) static Nimbusvoidstatic voidvoidprocessWorkerMetrics(WorkerMetrics metrics) voidrebalance(String topoName, RebalanceOptions options) voidrmDependencyJarsInTopology(String topoId) voidrmTopologyKeys(String topoId) voidSend supervisor local worker heartbeat when a supervisor is unreachablevoidSend supervisor worker heartbeats for a specific supervisorvoidsetAuthorizationHandler(IAuthorizer authorizationHandler) voidsetBlobMeta(String key, SettableBlobMeta meta) voidsetLogConfig(String topoId, LogConfig config) static voidsetUpAckerExecutorConfigs(String topoName, Map<String, Object> totalConfToSave, Map<String, Object> totalConf, int estimatedNumWorker) voidsetWorkerProfiler(String topoId, ProfileRequest profileRequest) voidshutdown()voidsubmitTopology(String name, String uploadedJarLocation, String jsonConf, StormTopology topology) voidsubmitTopologyWithOpts(String topoName, String uploadedJarLocation, String jsonConf, StormTopology topology, SubmitOptions options) topoIdsToClean(IStormClusterState state, BlobStore store, Map<String, Object> conf) topologiesOnSupervisor(Map<String, Assignment> assignments, String supervisorId) intupdateBlobReplication(String key, int replication) voiduploadBlobChunk(String session, ByteBuffer chunk) voiduploadChunk(String location, ByteBuffer chunk) voiduploadNewCredentials(String topoName, Credentials credentials)
-
Field Details
-
ZK_ACLS
-
MIN_VERSION_SUPPORT_RPC_HEARTBEAT
-
NIMBUS_SUBJECT
-
-
Constructor Details
-
Nimbus
public Nimbus(Map<String, Object> conf, INimbus inimbus, StormMetricsRegistry metricsRegistry) throws Exception- Throws:
Exception
-
Nimbus
public Nimbus(Map<String, Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry) throws Exception- Throws:
Exception
-
Nimbus
public Nimbus(Map<String, Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, TopoCache topoCache, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry) throws Exception- Throws:
Exception
-
-
Method Details
-
getNimbusAcls
-
getVersionForKey
public static int getVersionForKey(String key, NimbusInfo nimbusInfo, org.apache.storm.shade.org.apache.curator.framework.CuratorFramework zkClient) throws KeyNotFoundException - Throws:
KeyNotFoundException
-
topoIdsToClean
-
cleanInbox
Deletes jar files in dirLoc older than seconds.- Parameters:
dirLoc- the location to look in for fileseconds- how old is too old and should be deleted
-
topologiesOnSupervisor
-
launchServer
- Throws:
Exception
-
launch
- Throws:
Exception
-
main
- Throws:
Exception
-
setAuthorizationHandler
-
getHeartbeatsCache
-
getIdToExecutors
-
addSupervisor
Used for local cluster.- Parameters:
supervisor-Supervisor
-
checkAuthorization
public void checkAuthorization(String topoName, Map<String, Object> topoConf, String operation) throws AuthorizationException- Throws:
AuthorizationException
-
checkAuthorization
public void checkAuthorization(String topoName, Map<String, Object> topoConf, String operation, ReqContext context) throws AuthorizationException- Throws:
AuthorizationException
-
filterAuthorized
public Set<String> filterAuthorized(String operation, Collection<String> topoIds) throws NotAliveException, AuthorizationException, IOException -
rmDependencyJarsInTopology
-
rmTopologyKeys
-
forceDeleteTopoDistDir
- Throws:
IOException
-
doCleanup
public void doCleanup()Cleanup topologies and Jars. -
awaitLeadership
- Throws:
InterruptedException
-
submitTopology
public void submitTopology(String name, String uploadedJarLocation, String jsonConf, StormTopology topology) throws AlreadyAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
submitTopologyin interfaceNimbus.Iface- Throws:
AlreadyAliveExceptionInvalidTopologyExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
submitTopologyWithOpts
public void submitTopologyWithOpts(String topoName, String uploadedJarLocation, String jsonConf, StormTopology topology, SubmitOptions options) throws AlreadyAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
submitTopologyWithOptsin interfaceNimbus.Iface- Throws:
AlreadyAliveExceptionInvalidTopologyExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
setUpAckerExecutorConfigs
-
killTopology
public void killTopology(String name) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
killTopologyin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
killTopologyWithOpts
public void killTopologyWithOpts(String topoName, KillOptions options) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
killTopologyWithOptsin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
activate
public void activate(String topoName) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
activatein interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
deactivate
public void deactivate(String topoName) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
deactivatein interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
rebalance
public void rebalance(String topoName, RebalanceOptions options) throws NotAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
rebalancein interfaceNimbus.Iface- Throws:
NotAliveExceptionInvalidTopologyExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
setLogConfig
- Specified by:
setLogConfigin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
getLogConfig
- Specified by:
getLogConfigin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
debug
public void debug(String topoName, String componentId, boolean enable, double samplingPercentage) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.IfaceEnable/disable logging the tuples generated in topology via an internal EventLogger bolt. The component name is optional and if null or empty, the debug flag will apply to the entire topology. The 'samplingPercentage' will limit loggging to a percentage of generated tuples.- Specified by:
debugin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
setWorkerProfiler
public void setWorkerProfiler(String topoId, ProfileRequest profileRequest) throws org.apache.storm.thrift.TException - Specified by:
setWorkerProfilerin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
getComponentPendingProfileActions
public List<ProfileRequest> getComponentPendingProfileActions(String id, String componentId, ProfileAction action) throws org.apache.storm.thrift.TException - Specified by:
getComponentPendingProfileActionsin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
uploadNewCredentials
public void uploadNewCredentials(String topoName, Credentials credentials) throws NotAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
uploadNewCredentialsin interfaceNimbus.Iface- Throws:
NotAliveExceptionInvalidTopologyExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
beginCreateBlob
public String beginCreateBlob(String key, SettableBlobMeta meta) throws AuthorizationException, KeyAlreadyExistsException, org.apache.storm.thrift.TException - Specified by:
beginCreateBlobin interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyAlreadyExistsExceptionorg.apache.storm.thrift.TException
-
beginUpdateBlob
public String beginUpdateBlob(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException - Specified by:
beginUpdateBlobin interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionorg.apache.storm.thrift.TException
-
uploadBlobChunk
public void uploadBlobChunk(String session, ByteBuffer chunk) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
uploadBlobChunkin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
finishBlobUpload
public void finishBlobUpload(String session) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
finishBlobUploadin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
cancelBlobUpload
public void cancelBlobUpload(String session) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
cancelBlobUploadin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getBlobMeta
public ReadableBlobMeta getBlobMeta(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException - Specified by:
getBlobMetain interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionorg.apache.storm.thrift.TException
-
setBlobMeta
public void setBlobMeta(String key, SettableBlobMeta meta) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException - Specified by:
setBlobMetain interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionorg.apache.storm.thrift.TException
-
beginBlobDownload
public BeginDownloadResult beginBlobDownload(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException - Specified by:
beginBlobDownloadin interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionorg.apache.storm.thrift.TException
-
downloadBlobChunk
public ByteBuffer downloadBlobChunk(String session) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
downloadBlobChunkin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
deleteBlob
public void deleteBlob(String key) throws AuthorizationException, KeyNotFoundException, IllegalStateException, org.apache.storm.thrift.TException - Specified by:
deleteBlobin interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionIllegalStateExceptionorg.apache.storm.thrift.TException
-
listBlobs
- Specified by:
listBlobsin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
getBlobReplication
public int getBlobReplication(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException - Specified by:
getBlobReplicationin interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionorg.apache.storm.thrift.TException
-
updateBlobReplication
public int updateBlobReplication(String key, int replication) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException - Specified by:
updateBlobReplicationin interfaceNimbus.Iface- Throws:
AuthorizationExceptionKeyNotFoundExceptionorg.apache.storm.thrift.TException
-
createStateInZookeeper
- Specified by:
createStateInZookeeperin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
beginFileUpload
- Specified by:
beginFileUploadin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
uploadChunk
public void uploadChunk(String location, ByteBuffer chunk) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
uploadChunkin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
finishFileUpload
public void finishFileUpload(String location) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
finishFileUploadin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
downloadChunk
public ByteBuffer downloadChunk(String id) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
downloadChunkin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getNimbusConf
- Specified by:
getNimbusConfin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologyInfo
public TopologyInfo getTopologyInfo(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologyInfoin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologyInfoByName
- Specified by:
getTopologyInfoByNamein interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
getTopologyInfoByNameWithOpts
public TopologyInfo getTopologyInfoByNameWithOpts(String name, GetInfoOptions options) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologyInfoByNameWithOptsin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologyInfoWithOpts
public TopologyInfo getTopologyInfoWithOpts(String topoId, GetInfoOptions options) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologyInfoWithOptsin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologyPageInfo
public TopologyPageInfo getTopologyPageInfo(String topoId, String window, boolean includeSys) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologyPageInfoin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getSupervisorPageInfo
public SupervisorPageInfo getSupervisorPageInfo(String superId, String host, boolean includeSys) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getSupervisorPageInfoin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getComponentPageInfo
public ComponentPageInfo getComponentPageInfo(String topoId, String componentId, String window, boolean includeSys) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getComponentPageInfoin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologyConf
public String getTopologyConf(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologyConfin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopology
public StormTopology getTopology(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.IfaceReturns the compiled topology that contains ackers and metrics consumsers. CompareNimbus.Iface.getUserTopology(String id).- Specified by:
getTopologyin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getUserTopology
public StormTopology getUserTopology(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.IfaceReturns the user specified topology as submitted originally. CompareNimbus.Iface.getTopology(String id).- Specified by:
getUserTopologyin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologyHistory
public TopologyHistoryInfo getTopologyHistory(String user) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologyHistoryin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getClusterInfo
public ClusterSummary getClusterInfo() throws AuthorizationException, org.apache.storm.thrift.TException- Specified by:
getClusterInfoin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologySummaries
public List<TopologySummary> getTopologySummaries() throws AuthorizationException, org.apache.storm.thrift.TException- Specified by:
getTopologySummariesin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologySummaryByName
public TopologySummary getTopologySummaryByName(String name) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologySummaryByNamein interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getTopologySummary
public TopologySummary getTopologySummary(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologySummaryin interfaceNimbus.Iface- Throws:
NotAliveExceptionAuthorizationExceptionorg.apache.storm.thrift.TException
-
getLeader
- Specified by:
getLeaderin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
isTopologyNameAllowed
public boolean isTopologyNameAllowed(String name) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
isTopologyNameAllowedin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getOwnerResourceSummaries
public List<OwnerResourceSummary> getOwnerResourceSummaries(String owner) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getOwnerResourceSummariesin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
getSupervisorAssignments
public SupervisorAssignments getSupervisorAssignments(String nodeId) throws AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.IfaceGet assigned assignments for a specific supervisor- Specified by:
getSupervisorAssignmentsin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
sendSupervisorWorkerHeartbeats
public void sendSupervisorWorkerHeartbeats(SupervisorWorkerHeartbeats heartbeats) throws AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.IfaceSend supervisor worker heartbeats for a specific supervisor- Specified by:
sendSupervisorWorkerHeartbeatsin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
sendSupervisorWorkerHeartbeat
public void sendSupervisorWorkerHeartbeat(SupervisorWorkerHeartbeat hb) throws AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.IfaceSend supervisor local worker heartbeat when a supervisor is unreachable- Specified by:
sendSupervisorWorkerHeartbeatin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-
shutdown
public void shutdown()- Specified by:
shutdownin interfaceShutdownable
-
isWaiting
public boolean isWaiting()- Specified by:
isWaitingin interfaceDaemonCommon
-
processWorkerMetrics
- Specified by:
processWorkerMetricsin interfaceNimbus.Iface- Throws:
org.apache.storm.thrift.TException
-
isRemoteBlobExists
public boolean isRemoteBlobExists(String blobKey) throws AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.IfaceDecide if the blob is removed from cluster.- Specified by:
isRemoteBlobExistsin interfaceNimbus.Iface- Throws:
AuthorizationExceptionorg.apache.storm.thrift.TException
-