学习目标:
根据用户输入的检索条件,查询出对用的商品
首页的分类
搜索栏
这时我们要思考三个问题:
哪些字段需要分词
我们用哪些字段进行过滤
平台属性值
分类Id
品牌Id
哪些字段我们需要通过搜索查询出来。
以上分析的所有显示,以及分词,过滤的字段都应该在es中出现。Es中如何保存这些数据呢?
根据上述的字段描述,应该建立一个mappings对应的存上上述字段描述的信息!
根据以上制定出如下结构:mappings
Index:goods
document: properties
field: id,price,title…
ES中index默认是true。
注意:ik_max_word 中文词库必须有!
attrs:平台属性值的集合,主要用于平台属性值过滤。
nested:类型是一种特殊的对象object数据类型(specialised version of the object datatype ),允许对象数组彼此独立地进行索引和查询。
demo: 建立一个普通的index
如果linux 中有这个my_comment_index 先删除!DELETE /my_comment_index
步骤1:建立一个索引( 存储博客文章及其所有评论)
PUT my_comment_index/_doc/1
{
"title": "狂人日记",
"body": "《狂人日记》是一篇象征性和寓意很强的小说,当时,鲁迅对中国国民精神的麻木愚昧颇感痛切。",
"comments": [
{
"name": "张三",
"age": 34,
"rating": 8,
"comment": "非常棒的文章",
"commented_on": "30 Nov 2023"
},
{
"name": "李四",
"age": 38,
"rating": 9,
"comment": "文章非常好",
"commented_on": "25 Nov 2022"
},
{
"name": "王五",
"age": 33,
"rating": 7,
"comment": "手动点赞",
"commented_on": "20 Nov 2021"
}
]
}
如上所示,所以我们有一个文档描述了一个帖子和一个包含帖子上所有评论的内部对象评论。 但是Elasticsearch搜索中的内部对象并不像我们期望的那样工作。
步骤2 : 执行查询
GET /my_comment_index/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"comments.name": "李四"
}
},
{
"match": {
"comments.age": 34
}
}
]
}
}
}
查询结果:居然正常的响应结果了
原因分析:comments字段默认的数据类型是Object,故我们的文档内部存储为:
{ "title": [ 狂人日记], "body": [ 《狂人日记》是一篇象征性和寓意很强的小说,当时... ], "comments.name": [ 张三, 李四, 王五 ], "comments.comment": [ 非常棒的文章,文章非常好,王五,... ], "comments.age": [ 33, 34, 38 ], "comments.rating": [ 7, 8, 9 ] }
我们可以清楚地看到,comments.name和comments.age之间的关系已丢失。这就是为什么我们的文档匹配李四和34的查询。
步骤3:删除当前索引
DELETE /my_comment_index
步骤4:建立一个nested 类型的(comments字段映射为nested类型,而不是默认的object类型)
PUT my_comment_index
{
"mappings": {
"properties": {
"comments": {
"type": "nested"
}
}
}
}
PUT my_comment_index/_doc/1
{
"title": "狂人日记",
"body": "《狂人日记》是一篇象征性和寓意很强的小说,当时,鲁迅对中国国民精神的麻木愚昧颇感痛切。",
"comments": [
{
"name": "张三",
"age": 34,
"rating": 8,
"comment": "非常棒的文章",
"commented_on": "30 Nov 2023"
},
{
"name": "李四",
"age": 38,
"rating": 9,
"comment": "文章非常好",
"commented_on": "25 Nov 2022"
},
{
"name": "王五",
"age": 33,
"rating": 7,
"comment": "手动点赞",
"commented_on": "20 Nov 2021"
}
]
}
重新执行步骤1,使用nested 查询
GET /my_comment_index/_search
{
"query": {
"nested": {
"path": "comments",
"query": {
"bool": {
"must": [
{
"match": {
"comments.name": "李四"
}
},
{
"match": {
"comments.age": 34
}
}
]
}
}
}
}
}
结果发现没有返回任何的文档,这是何故?
当将字段设置为nested 嵌套对象将数组中的每个对象索引为单独的隐藏文档,这意味着可以独立于其他对象查询每个嵌套对象。文档的内部表示:
{ {
"comments.name": [ 张三], "comments.comment": [ 非常棒的文章 ], "comments.age": [ 38 ], "comments.rating": [ 9 ]
}, {
"comments.name": [ 李四], "comments.comment": [ 文章非常好 ], "comments.age": [ 34 ], "comments.rating": [ 8 ]
}, {
"comments.name": [ 王五], "comments.comment": [手动点赞], "comments.age": [ 33 ], "comments.rating": [ 7 ]
}, {
"title": [ 狂人日记 ], "body": [ 《狂人日记》是一篇象征性和寓意很强的小说,当时,鲁迅对中国... ]
} }
每个内部对象都在内部存储为单独的隐藏文档。 这保持了他们的领域之间的关系。
在gmall-service
模块下搭建搜索模块:service-list
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>gmall-service</artifactId>
<groupId>com.atguigu.gmall</groupId>
<version>1.0</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>service-list</artifactId>
<dependencies>
<dependency>
<groupId>com.atguigu.gmall</groupId>
<artifactId>service-product-client</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
</dependencies>
</project>
说明:
引入service-product-client模块
引入spring-boot-starter-data-elasticsearch依赖
在父工程中gmall-parent
模块pom.xml中properties节点中指定client版本跟ES服务端版本一致
<elasticsearch.version>7.8.0</elasticsearch.version>
package com.atguigu.gmall;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;
import org.springframework.cloud.openfeign.EnableFeignClients;
@SpringBootApplication(exclude = DataSourceAutoConfiguration.class)
@EnableDiscoveryClient
@EnableFeignClients
public class ListApp {
public static void main(String[] args) {
SpringApplication.run(ListApp.class, args);
}
}
bootstrap.properties
spring.application.name=service-list
spring.profiles.active=dev
spring.cloud.nacos.discovery.server-addr=192.168.200.128:8848
spring.cloud.nacos.config.server-addr=192.168.200.128:8848
spring.cloud.nacos.config.prefix=${spring.application.name}
spring.cloud.nacos.config.file-extension=yaml
spring.cloud.nacos.config.shared-configs[0].data-id=common.yaml
说明:Nacos配置文件中添加es配置信息
说明:在gmall-model
模块中已有商品文档实体类,以及平台属性实体类跟ES索引库mapping简历映射
商品文档实体类
package com.atguigu.gmall.list.model;
import lombok.Data;
import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.DateFormat;
import org.springframework.data.elasticsearch.annotations.Document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;
import java.util.Date;
import java.util.List;
import java.util.Objects;
// Index = goods , Type = info es 7.8.0 逐渐淡化type! 修改!
// es 的分片,副本是为了保证高可用!
@Data
@Document(indexName = "goods" , shards = 3,replicas = 2)
public class Goods {
// 商品Id skuId
@Id
private Long id;
@Field(type = FieldType.Keyword, index = false)
private String defaultImg;
// es 中能分词的字段,这个字段数据类型必须是 text!
@Field(type = FieldType.Text, analyzer = "ik_max_word")
private String title;
@Field(type = FieldType.Double)
private Double price;
// @Field(type = FieldType.Date) 6.8.1
@Field(type = FieldType.Date,format = DateFormat.custom,pattern = "yyyy-MM-dd HH:mm:ss")
private Date createTime; // 新品
@Field(type = FieldType.Long)
private Long tmId;
@Field(type = FieldType.Keyword)
private String tmName;
@Field(type = FieldType.Keyword)
private String tmLogoUrl;
@Field(type = FieldType.Long)
private Long category1Id;
@Field(type = FieldType.Keyword)
private String category1Name;
@Field(type = FieldType.Long)
private Long category2Id;
@Field(type = FieldType.Keyword)
private String category2Name;
@Field(type = FieldType.Long)
private Long category3Id;
@Field(type = FieldType.Keyword)
private String category3Name;
// 商品的热度! 我们将商品被用户点查看的次数越多,则说明热度就越高!
@Field(type = FieldType.Long)
private Long hotScore = 0L;
// 平台属性集合对象
// Nested 支持嵌套查询 允许对象数组彼此独立检索和查询
@Field(type = FieldType.Nested)
private List<SearchAttr> attrs;
}
销售属性实体类
package com.atguigu.gmall.model.list;
import lombok.Data;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;
@Data
public class SearchAttr {
// 平台属性Id
@Field(type = FieldType.Long)
private Long attrId;
// 平台属性值名称
@Field(type = FieldType.Keyword)
private String attrValue;
// 平台属性名
@Field(type = FieldType.Keyword)
private String attrName;
}
package com.atguigu.gmall.list.controller;
import com.atguigu.gmall.common.result.Result;
import com.atguigu.gmall.list.model.Goods;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequestMapping("api/list")
public class ListApiController {
@Autowired
private ElasticsearchRestTemplate restTemplate;
/** 创建商品索引库
* @return
*/
@GetMapping("inner/createIndex")
public Result createIndex() {
restTemplate.createIndex(Goods.class);
restTemplate.putMapping(Goods.class);
return Result.ok();
}
}
在浏览器运行:
http://localhost:8203/api/list/inner/createIndex
通过kibana查看mapping
重点:attrs 数据类型必须是nested !
构建goods数据模型分析
Sku基本信息(详情业务已封装了接口)
Sku分类信息(详情业务已封装了接口)
Sku的品牌信息(无)
Sku对应的平台属性(详情业务已封装了接口)
YAPI接口地址:http://192.168.200.128:3000/project/11/interface/api/643
service-product
商品微服务模块中增加查询品牌RestFul接口实现
控制器ProductApiController
@Autowired
private BaseTrademarkService baseTrademarkService;
/**
* 根据品牌ID查询品牌信息
*
* @param tmId 品牌ID
* @return
*/
@GetMapping("/inner/getTrademark/{tmId}")
public BaseTrademark getTrademarkById(@PathVariable("tmId") Long tmId) {
BaseTrademark trademark = baseTrademarkService.getById(tmId);
return trademark;
}
在service-product-client
中ProductFeignClient提供Feign API接口
/**
* 根据品牌ID查询品牌信息
*
* @param tmId 品牌ID
* @return
*/
@GetMapping("/api/product/inner/getTrademark/{tmId}")
public BaseTrademark getTrademarkById(@PathVariable("tmId") Long tmId);
服务降级类
@Override
public BaseTrademark getTrademark(Long tmId) {
return null;
}
YAPI接口文档:
service-list
模块ListApiController 处理上下架请求
@Autowired
private SearchService searchService;
/**
* 测试接口,商品文档对象录入索引
* @param skuId
* @return
*/
@GetMapping("/inner/upperGoods/{skuId}")
public Result upperGoods(@PathVariable("skuId") Long skuId){
searchService.upperGoods(skuId);
return Result.ok();
}
/**
* 测试接口,商品文档删除
* @param skuId
* @return
*/
@GetMapping("/inner/lowerGoods/{skuId}")
public Result lowerGoods(@PathVariable("skuId") Long skuId){
searchService.lowerGoods(skuId);
return Result.ok();
}
SearchService
package com.atguigu.gmall.list.service;
public interface SearchService {
/**
* 商品上架
* @param skuId
*/
void upperGoods(Long skuId);
/**
* 商品下架
* @param skuId
*/
void lowerGoods(Long skuId);
}
SearchServiceImpl
package com.atguigu.gmall.list.service.impl;
import com.alibaba.fastjson.JSON;
import com.atguigu.gmall.list.model.Goods;
import com.atguigu.gmall.list.model.SearchAttr;
import com.atguigu.gmall.list.service.SearchService;
import com.atguigu.gmall.product.client.ProductFeignClient;
import com.atguigu.gmall.product.model.BaseAttrInfo;
import com.atguigu.gmall.product.model.BaseCategoryView;
import com.atguigu.gmall.product.model.BaseTrademark;
import com.atguigu.gmall.product.model.SkuInfo;
import lombok.extern.slf4j.Slf4j;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.xcontent.XContentType;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.util.CollectionUtils;
import java.io.IOException;
import java.util.Date;
import java.util.List;
import java.util.stream.Collectors;
/**
* @author: atguigu
* @create: 2023-01-06 15:37
*/
@Slf4j
@Service
@SuppressWarnings("all")
public class SearchServiceImpl implements SearchService {
@Autowired
private ProductFeignClient productFeignClient;
@Autowired
private RestHighLevelClient restHighLevelClient;
/**
* 常量:索引库名称
*/
private static final String INDEX_NAME = "goods";
/**
* 1.远程调用商品微服务获取商品相关信息
* 2.封装索引库商品文档Goods
* 3.调用ES将文档对象存入索引库
*
* @param skuId
*/
@Override
public void upperGoods(Long skuId) {
try {
//1.创建索引库文档对象:Goods
Goods goods = new Goods();
//2.封装商品文档对象Goods中的属性赋值
//2.1 根据SkuID远程查询SkuInfo商品信息
SkuInfo skuInfo = productFeignClient.getSkuInfo(skuId);
if (skuInfo != null) {
goods.setId(skuInfo.getId());
goods.setTitle(skuInfo.getSkuName());
goods.setCategory3Id(skuInfo.getCategory3Id());
goods.setPrice(skuInfo.getPrice().doubleValue());
goods.setDefaultImg(skuInfo.getSkuDefaultImg());
goods.setCreateTime(new Date());
goods.setCreatedDate(skuInfo.getCreateTime());
//2.2 根据skuInfo的品牌ID查询品牌信息
BaseTrademark trademark = productFeignClient.getTrademarkById(skuInfo.getTmId());
if (trademark != null) {
goods.setTmId(trademark.getId());
goods.setTmName(trademark.getTmName());
goods.setTmLogoUrl(trademark.getLogoUrl());
}
//2.3 根据分类ID查询分类信息
BaseCategoryView categoryView = productFeignClient.getCategoryView(skuInfo.getCategory3Id());
if (categoryView != null) {
goods.setCategory1Id(categoryView.getCategory1Id());
goods.setCategory1Name(categoryView.getCategory1Name());
goods.setCategory2Id(categoryView.getCategory2Id());
goods.setCategory2Name(categoryView.getCategory2Name());
goods.setCategory3Id(categoryView.getCategory3Id());
goods.setCategory3Name(categoryView.getCategory3Name());
}
//2.4 根据skuID查询平台属性以及值
List<BaseAttrInfo> attrList = productFeignClient.getAttrList(skuId);
if (!CollectionUtils.isEmpty(attrList)) {
List<SearchAttr> attrs = attrList.stream().map(baseAttrInfo -> {
SearchAttr searchAttr = new SearchAttr();
searchAttr.setAttrId(baseAttrInfo.getId());
searchAttr.setAttrName(baseAttrInfo.getAttrName());
searchAttr.setAttrValue(baseAttrInfo.getAttrValueList().get(0).getValueName());
return searchAttr;
}).collect(Collectors.toList());
goods.setAttrs(attrs);
}
//3.将索引库文档对象存入索引库ES
//3.1 创建IndexRequest对象 封装索引库名称 文档ID 请求JSON
IndexRequest indexRequest = new IndexRequest(INDEX_NAME)
.id(skuInfo.getId().toString())
.source(JSON.toJSONString(goods), XContentType.JSON);
//3.2 执行保存
restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
}
} catch (IOException e) {
e.printStackTrace();
log.error("商品上架失败:{}", e);
}
}
/**
* 删除商品文档
*
* @param skuId
*/
@Override
public void lowerGoods(Long skuId) {
try {
DeleteRequest request = new DeleteRequest(
INDEX_NAME,
skuId.toString());
restHighLevelClient.delete(request, RequestOptions.DEFAULT);
} catch (IOException e) {
e.printStackTrace();
log.error("商品下架失败:{}", e);
}
}
}
添加数据
通过kibana查看数据
说明:后期学习了MQ,我们可以根据后台系统添加和修改等操作,发送mq消息自动上下架商品
http://localhost:8203/api/list/inner/upperGoods/21
http://localhost:8203/api/list/inner/lowerGoods/21
搜索商品时,后面我们会根据热点排序,何时更新热点?我们在获取商品详情时调用更新
YAPI接口地址:http://192.168.200.128:3000/project/11/interface/api/739
ListController
/**
* 更新商品的热度排名分值
* @param skuId
* @return
*/
@GetMapping("/inner/incrHotScore/{skuId}")
public Result incrHotScore(@PathVariable("skuId") Long skuId){
searchService.incrHotScore(skuId);
return Result.ok();
}
SearchService
/**
* 更新商品的热度排名分值
* @param skuId
* @return
*/
void incrHotScore(Long skuId);
SearchServiceImpl
/**
* 更新商品的热度排名分值
*
* @param skuId
* @return
*/
@Override
public void incrHotScore(Long skuId) {
try {
//1.更新Redis缓存中商品热点排名分值 分值+1
//1.1 构建ZSet排名Key
String hotKey = "hotScore";
//1.2 调用自增分值+1方法为查询商品增加分值
String skuIdStr = skuId.toString();
Double score = redisTemplate.opsForZSet().incrementScore(hotKey, skuIdStr, 1);
//2.满足更新索引库条件后再更新文档中分值
if (score % 10 == 0) {
//2.0 根据文档ID查询商品文档对象
Map<String, Object> jsonMap = new HashMap<>();
//注意:索引库中存储是Long类型 将分值有Double转为Long
jsonMap.put("hotScore", score.longValue());
//2.1 创建更新文档对象 UpdateRequest
UpdateRequest updateRequest = new UpdateRequest(INDEX_NAME, skuIdStr);
updateRequest.doc(jsonMap);
//2.2 更新文档
restHighLevelClient.update(
updateRequest, RequestOptions.DEFAULT);
}
} catch (IOException e) {
e.printStackTrace();
}
}
在gmall-client
模块下搭建:service-list-client模块。搭建方式如service-item-client
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>gmall-client</artifactId>
<groupId>com.atguigu.gmall</groupId>
<version>1.0</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>service-list-client</artifactId>
</project>
提供远程调用的Feign接口
package com.atguigu.gmall.list.client;
import com.atguigu.gmall.common.result.Result;
import com.atguigu.gmall.list.client.impl.ListDegradeFeignClient;
import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
@FeignClient(value = "service-list", fallback = ListDegradeFeignClient.class)
public interface ListFeignClient {
/**
* 更新商品incrHotScore
*
* @param skuId
* @return
*/
@GetMapping("/api/list/inner/incrHotScore/{skuId}")
Result incrHotScore(@PathVariable("skuId") Long skuId);
}
服务降级类
package com.atguigu.gmall.list.client.impl;
import com.atguigu.gmall.common.result.Result;
import com.atguigu.gmall.list.client.ListFeignClient;
import org.springframework.stereotype.Component;
@Component
public class ListDegradeFeignClient implements ListFeignClient {
@Override
public Result incrHotScore(Long skuId) {
return null;
}
}
在service-item
模块pom.xml中引入依赖
<dependency>
<groupId>com.atguigu.gmall</groupId>
<artifactId>service-list-client</artifactId>
<version>1.0</version>
</dependency>
接口调用,更新service-item
模块中ItemServiceImpl
汇总商品信息方法:getBySkuId
package com.atguigu.gmall.item.service.impl;
import com.alibaba.fastjson.JSON;
import com.atguigu.gmall.common.constant.RedisConst;
import com.atguigu.gmall.common.result.Result;
import com.atguigu.gmall.item.service.ItemService;
import com.atguigu.gmall.list.client.ListFeignClient;
import com.atguigu.gmall.product.client.ProductFeignClient;
import com.atguigu.gmall.product.model.*;
import org.redisson.api.RBloomFilter;
import org.redisson.api.RedissonClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.math.BigDecimal;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.stream.Collectors;
@Service
public class ItemServiceImpl implements ItemService {
/**
* 通过调用service-product-client模块中提供的远程调用API接口调用 service-product 提供的服务
*/
@Autowired
private ProductFeignClient productFeignClient;
@Autowired
private RedissonClient redissonClient;
@Autowired
private ThreadPoolExecutor threadPoolExecutor;
/**
* 通过调用service-list-client模块中提供的远程调用API接口调用 service-list 提供的服务
*/
@Autowired
private ListFeignClient listFeignClient;
@Override
public Map<String, Object> getBySkuId(Long skuId) {
// 声明对象
Map<String, Object> result = new HashMap<>();
// 省略1-6。。。异步多线程获取其他汇总新..
//7.根据spuID查询销售属性属性值对应sku信息Map {"销售属性1|销售属性2":"skuId"} TODO 注意要将map转为JSON
CompletableFuture<Void> valuesSkuJsonCompletableFuture = skuInfoCompletableFuture.thenAcceptAsync(skuInfo -> {
Map map = productFeignClient.getSkuValueIdsMap(skuInfo.getSpuId());
if (!CollectionUtils.isEmpty(map)) {
//将map转为JSON字符串
result.put("valuesSkuJson", JSON.toJSONString(map));
}
}, threadPoolExecutor);
//8.远程调用搜索微服务,更新ES索引库中商品文档热门分值
CompletableFuture<Void> incrHotScoreCompletableFuture = CompletableFuture.runAsync(() -> {
listFeignClient.incrHotScore(skuId);
}, threadPoolExecutor);
//9.将以上八个任务全部并行执行,执行完所有任务才返回
CompletableFuture.allOf(skuInfoCompletableFuture,
categoryViewCompletableFuture,
priceCompletableFuture,
spuPosterListCompletableFuture,
attrListCompletableFuture,
spuSaleAttrListCompletableFuture,
valuesSkuJsonCompletableFuture,
incrHotScoreCompletableFuture).join();
return result;
}
}