我要评论1.什么是dag？
有向无环图(directed acyclic graph)，简称dag，是一种有向图，其中没有从节点出发经过若干条边后再回到该节点的路径。换句话说，dag中不存在环路。这种数据结构常用于表示并解决具有依赖关系的问题。
dag的特性
- 首先，dag中的节点可以有入度和出度。节点的入度是指指向该节点的边的数量，而节点的出度是指由该节点指向其他节点的边的数量。在dag中，节点的入度可以是0或正整数，而出度可以是0或正整数，但不能同时为负数。
- dag的另一个重要性质是存在一个或多个拓扑排序。拓扑排序是dag中节点的线性排列，满足任意一条有向边的起点在排序中都位于终点之前。可以使用深度优先搜索(dfs)或宽度优先搜索(bfs)算法来生成拓扑排序。
dag的应用
- 任务调度
- 编译器优化
- 数据流分析
- 电路设计
2.如何加速spring bean初始化？
在spring框架中进行dag（有向无环图）分析以实现并行初始化，可以有效提升应用程序启动的性能。通常情况下，spring应用程序的bean是按依赖顺序初始化的，而通过dag分析可以识别出哪些bean之间没有依赖关系，并行初始化这些bean可以减少启动时间。以下是实现思路：
1. 识别依赖关系，构建dag
首先需要识别spring bean之间的依赖关系。可以通过@dependson注解、构造器注入或@autowired等方式获取bean依赖。具体步骤：
- 遍历spring上下文中的所有bean定义。
- 根据bean的依赖关系构建dag，节点代表bean，边表示依赖关系。
spring的applicationcontext提供了getbeandefinitionnames()方法可以列出所有的bean，通过beandefinition可以分析出依赖。
2. 拓扑排序
通过拓扑排序（topological sorting）对dag进行排序，以确保bean按依赖顺序初始化。拓扑排序可以确定哪些bean可以并行初始化，哪些bean必须在某些bean之后初始化。 使用算法如kahn’s algorithm或dfs找到所有没有依赖的bean（入度为0的节点），这些节点可以并行初始化。
3. 并行初始化bean
在完成拓扑排序后，使用多线程来并行初始化可以同时启动的bean。可以使用java的executorservice或类似的线程池机制来管理并发的bean初始化过程。 步骤：
- 针对所有入度为0的节点，启动一个线程来初始化它们。
- 当某个bean初始化完成后，减少它所依赖的其他bean的入度值。
- 当某个bean的入度为0时，可以在另一个线程中启动它的初始化。
4. spring integration
可以通过beanfactorypostprocessor或applicationcontextinitializer来挂钩到spring的初始化流程中，分析bean之间的依赖关系，并将并行化初始化逻辑集成到spring容器的启动过程中。 具体方法：
- 实现
beanfactorypostprocessor，在bean初始化之前分析bean的依赖并构建dag。 - 在bean初始化阶段，使用多线程并行处理独立的bean。
3.代码工程
整体的依赖关系如下：
实验目标
实现自定义bean并行化加载
pom.xml
<?xml version="1.0" encoding="utf-8"?>
<project xmlns="http://maven.apache.org/pom/4.0.0"
xmlns:xsi="http://www.w3.org/2001/xmlschema-instance"
xsi:schemalocation="http://maven.apache.org/pom/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupid>org.springframework.boot</groupid>
<artifactid>spring-boot-starter-parent</artifactid>
<version>3.2.1</version>
</parent>
<modelversion>4.0.0</modelversion>
<artifactid>dag</artifactid>
<properties>
<maven.compiler.source>17</maven.compiler.source>
<maven.compiler.target>17</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupid>org.springframework.boot</groupid>
<artifactid>spring-boot-starter-web</artifactid>
</dependency>
<dependency>
<groupid>org.springframework.boot</groupid>
<artifactid>spring-boot-autoconfigure</artifactid>
</dependency>
<dependency>
<groupid>org.springframework.boot</groupid>
<artifactid>spring-boot-starter-test</artifactid>
<scope>test</scope>
</dependency>
<dependency>
<groupid>org.jgrapht</groupid>
<artifactid>jgrapht-core</artifactid>
<version>1.5.1</version>
</dependency>
</dependencies>
<build>
<pluginmanagement>
<plugins>
<plugin>
<groupid>org.apache.maven.plugins</groupid>
<artifactid>maven-compiler-plugin</artifactid>
<version>3.8.1</version>
<configuration>
<fork>true</fork>
<failonerror>false</failonerror>
</configuration>
</plugin>
<plugin>
<groupid>org.apache.maven.plugins</groupid>
<artifactid>maven-surefire-plugin</artifactid>
<version>2.22.2</version>
<configuration>
<forkcount>0</forkcount>
<failifnotests>false</failifnotests>
</configuration>
</plugin>
</plugins>
</pluginmanagement>
</build>
</project>
bean初始化
package com.et.config;
import org.jgrapht.graph.defaultedge;
import org.jgrapht.graph.directedacyclicgraph;
import org.springframework.beans.beansexception;
import org.springframework.beans.factory.config.beandefinition;
import org.springframework.beans.factory.config.beanfactorypostprocessor;
import org.springframework.beans.factory.config.configurablelistablebeanfactory;
import org.springframework.stereotype.component;
import java.util.*;
import java.util.concurrent.completablefuture;
import java.util.concurrent.executorservice;
import java.util.concurrent.executors;
import java.util.stream.collectors;
@component
public class dagbeaninitializer implements beanfactorypostprocessor {
private final executorservice executorservice = executors.newfixedthreadpool(10);
@override
public void postprocessbeanfactory(configurablelistablebeanfactory beanfactory) throws beansexception {
map<string, beandefinition> beandefinitionmap = new hashmap<>();
for (string beanname : beanfactory.getbeandefinitionnames()) {
beandefinition beandefinition = beanfactory.getbeandefinition(beanname);
beandefinitionmap.put(beanname, beandefinition);
}
// build dag
directedacyclicgraph<string, defaultedge> dag = builddag(beandefinitionmap,beanfactory);
// bean layers
list<set<string>> layers = getbeansbylayer(dag);
system.out.println("layers:"+layers);
// init bean by layers
initializebeansinlayers(layers, beanfactory);
}
// dag bean
private directedacyclicgraph<string, defaultedge> builddag(map<string, beandefinition> beandefinitionmap, configurablelistablebeanfactory beanfactory) {
dependencyresolver resolver = new dependencyresolver(beanfactory);
directedacyclicgraph<string, defaultedge> dag = new directedacyclicgraph<>(defaultedge.class);
for (string beanname : beandefinitionmap.keyset()) {
if(shouldloadbean(beanname)) {
dag.addvertex(beanname);
string[] dependencies = beandefinitionmap.get(beanname).getdependson();
if (dependencies != null) {
for (string dependency : dependencies) {
dag.addedge(dependency, beanname);
}
}
// get @autowired dependencies
set<string> autowiredependencies = resolver.getalldependencies(beanname);
for (string autowiredependency : autowiredependencies) {
// convert beanname
string autowirebeanname = converttobeanname(autowiredependency);
dag.addvertex(autowirebeanname);
dag.addedge(autowirebeanname, beanname);
}
}
}
return dag;
}
private string converttobeanname(string classname) {
string simplename = classname.substring(classname.lastindexof('.') + 1);
return character.tolowercase(simplename.charat(0)) + simplename.substring(1);
}
private list<set<string>> getbeansbylayer(directedacyclicgraph<string,defaultedge> dag) {
list<set<string>> layers = new arraylist<>();
map<string, integer> indegree = new hashmap<>();
queue<string> queue = new linkedlist<>();
// init all nodes degree
for (string vertex : dag) {
int degree = dag.indegreeof(vertex);
indegree.put(vertex, degree);
if (degree == 0) {
queue.offer(vertex); //zero degree as the first layer
}
}
// bfs process everylayers
while (!queue.isempty()) {
set<string> currentlayer = new hashset<>();
int size = queue.size();
for (int i = 0; i < size; i++) {
string currentbean = queue.poll();
currentlayer.add(currentbean);
// iterator layers
for (string successor : getsuccessors(dag,currentbean)) {
indegree.put(successor, indegree.get(successor) - 1);
if (indegree.get(successor) == 0) {
queue.offer(successor); // add next layer when the degress is zero
}
}
}
layers.add(currentlayer);
}
return layers;
}
// get next node
private set<string> getsuccessors(directedacyclicgraph<string, defaultedge> dag, string vertex) {
// get outgoingedges
set<defaultedge> outgoingedges = dag.outgoingedgesof(vertex);
// find the next node
return outgoingedges.stream()
.map(edge -> dag.getedgetarget(edge))
.collect(collectors.toset());
}
// init beans by layer
private void initializebeansinlayers(list<set<string>> layers, configurablelistablebeanfactory beanfactory) {
for (set<string> layer : layers) {
// beans of the same layer can be initialized in parallel
list<completablefuture<void>> futures = new arraylist<>();
for (string beanname : layer) {
// only load beans that wrote by yourself
if (shouldloadbean(beanname)) {
completablefuture<void> future = completablefuture.runasync(() -> {
try {
beanfactory.getbean(beanname); // init bean
} catch (exception e) {
system.err.println("failed to initialize bean: " + beanname);
e.printstacktrace();
}
}, executorservice);
futures.add(future);
}
}
//wait for all beans in the current layer to be initialized before initializing the next layer.
completablefuture<void> allof = completablefuture.allof(futures.toarray(new completablefuture[0]));
allof.join(); // make sure to be done on current layer
}
}
private boolean shouldloadbean(string beanname) {
return beanname.startswith("helloworldcontroller")
||beanname.startswith("serviceone")
||beanname.startswith("servicetwo")
||beanname.startswith("servicethree");
}
}
获取bean@autowired依赖
package com.et.config;
import org.springframework.beans.factory.config.beandefinition;
import org.springframework.beans.factory.config.configurablelistablebeanfactory;
import org.springframework.beans.factory.annotation.autowired;
import org.springframework.stereotype.component;
import java.lang.reflect.field;
import java.util.hashset;
import java.util.set;
@component
public class dependencyresolver {
private final configurablelistablebeanfactory beanfactory;
@autowired
public dependencyresolver(configurablelistablebeanfactory beanfactory) {
this.beanfactory = beanfactory;
}
public set<string> getalldependencies(string beanname) {
set<string> dependencies = new hashset<>();
// get bean definite
beandefinition beandefinition = beanfactory.getbeandefinition(beanname);
// reflect
try {
class<?> beanclass = class.forname(beandefinition.getbeanclassname());
field[] fields = beanclass.getdeclaredfields();
for (field field : fields) {
if (field.isannotationpresent(autowired.class)) {
dependencies.add(field.gettype().getname());
}
}
} catch (classnotfoundexception e) {
e.printstacktrace();
}
return dependencies;
}
}
controller
package com.et.controller;
import com.et.service.servicetwo;
import org.springframework.beans.factory.annotation.autowired;
import org.springframework.web.bind.annotation.requestmapping;
import org.springframework.web.bind.annotation.restcontroller;
import com.et.service.*;
import java.util.hashmap;
import java.util.map;
@restcontroller
public class helloworldcontroller {
@autowired
serviceone serviceone;
@autowired
servicetwo servicetwo;
@requestmapping("/hello")
public map<string, object> showhelloworld(){
map<string, object> map = new hashmap<>();
map.put("msg", "helloworld");
return map;
}
}
service
package com.et.service;
import org.springframework.stereotype.service;
/**
* @author liuhaihua
* @version 1.0
* @classname serviceone
* @description todo
* @date 2024/09/20/ 14:01
*/
@service
public class serviceone {
private void sayhi(){
system.out.println("this is service one sayhi");
}
}
package com.et.service;
import org.springframework.stereotype.service;
/**
* @author liuhaihua
* @version 1.0
* @classname serviceone
* @description todo
* @date 2024/09/20/ 14:01
*/
@service
public class servicethree {
private void sayhi(){
system.out.println("this is service three sayhi");
}
}
package com.et.service;
import org.springframework.beans.factory.annotation.autowired;
import org.springframework.stereotype.service;
/**
* @author liuhaihua
* @version 1.0
* @classname serviceone
* @description todo
* @date 2024/09/20/ 14:01
*/
@service
public class servicetwo {
@autowired
servicethree servicethree;
private void sayhi(){
system.out.println("this is service two sayhi");
}
}
只是一些关键代码
4.测试
启动spring boot工程，查看bean加载顺序如下
2024-09-20t15:51:27.081+08:00 info 33188 --- [ main] com.et.demoapplication : starting demoapplication using java 17.0.9 with pid 33188 (d:\ideaprojects\etframework\dag\target\classes started by dell in d:\ideaprojects\etframework) 2024-09-20t15:51:27.085+08:00 info 33188 --- [ main] com.et.demoapplication : no active profile set, falling back to 1 default profile: "default" layers:[[serviceone, servicethree], [servicetwo], [helloworldcontroller]] 2024-09-20t15:51:28.286+08:00 info 33188 --- [ main] o.s.b.w.embedded.tomcat.tomcatwebserver : tomcat initialized with port 8088 (http) 2024-09-20t15:51:28.297+08:00 info 33188 --- [ main] o.apache.catalina.core.standardservice : starting service [tomcat] 2024-09-20t15:51:28.297+08:00 info 33188 --- [ main] o.apache.catalina.core.standardengine : starting servlet engine: [apache tomcat/10.1.17] 2024-09-20t15:51:28.373+08:00 info 33188 --- [ main] o.a.c.c.c.[tomcat].[localhost].[/] : initializing spring embedded webapplicationcontext 2024-09-20t15:51:28.374+08:00 info 33188 --- [ main] w.s.c.servletwebserverapplicationcontext : root webapplicationcontext: initialization completed in 1198 ms 2024-09-20t15:51:28.725+08:00 info 33188 --- [ main] o.s.b.w.embedded.tomcat.tomcatwebserver : tomcat started on port 8088 (http) with context path '' 2024-09-20t15:51:28.732+08:00 info 33188 --- [ main] com.et.demoapplication
以上就是springboot利用dag加速spring beans初始化的方法示例的详细内容，更多关于springboot dag加速beans初始化的资料请关注代码网其它相关文章！
发表评论