Technology peripherals
AI
Exploring Siamese networks using contrastive loss for image similarity comparison
Exploring Siamese networks using contrastive loss for image similarity comparison
Apr 02, 2024 am 11:37 AM
Introduction
In the field of computer vision, accurately measuring image similarity is a critical task with a wide range of practical applications. From image search engines to facial recognition systems and content-based recommendation systems, the ability to effectively compare and find similar images is important. The Siamese network combined with contrastive loss provides a powerful framework for learning image similarity in a data-driven manner. In this blog post, we will dive into the details of Siamese networks, explore the concept of contrastive loss, and explore how these two components work together to create an effective image similarity model. First, the Siamese network consists of two identical subnetworks that share the same weights and parameters. Each sub-network encodes input images into feature vectors that capture key features of the image. We then use a contrastive loss to measure the similarity between the two input images. The contrastive loss is based on the Euclidean distance metric and adopts a restriction term to ensure that the distance between samples of the same class is smaller than the distance between samples of different classes. Through backpropagation and optimization algorithms, Siamese networks are able to automatically learn feature representations, making the ability to resemble images very important. The innovation of this model lies in its ability to learn relatively few samples in the training set and transfer them to the training set through transfer learning. Neural network architecture for measuring similarity between pairs of input samples. The term "Siamese" comes from the concept of a network architecture consisting of two identically structured Siamese neural networks that share the same set of weights. Each network processes one of the samples from the corresponding input and determines the similarity or dissimilarity between them by comparing their outputs. Each sample in the Siamese network deals with the similarity or dissimilarity between input samples from corresponding input samples. This similarity measure can be determined by comparing their outputs. Siamese networks are commonly used for recognition and verification tasks, such as face recognition, fingerprint recognition, and signature verification. It can automatically learn the similarities between input samples and make decisions based on training data. With a Siamese network, each network processes one of the corresponding input samples and compares their outputs to determine how similar or dissimilar they are.
- 共享網(wǎng)絡(luò)：共享網(wǎng)絡(luò)是Siamese架構(gòu)的核心組件。它負(fù)責(zé)從輸入樣本中提取有意義的特征表示。共享網(wǎng)絡(luò)包含神經(jīng)單元的層，例如卷積層或全連接層，用于處理輸入數(shù)據(jù)并生成固定長(zhǎng)度的embedding向量。通過在孿生網(wǎng)絡(luò)之間共享相同的權(quán)重，模型學(xué)會(huì)為相似的輸入提取相似的特征，從而實(shí)現(xiàn)有效的比較。
- 相似性度：一旦輸入由共享網(wǎng)絡(luò)處理，就會(huì)使用相似性度量來(lái)比較生成的embedding，并測(cè)量?jī)蓚€(gè)輸入之間的相似性或不相似性。相似度度量的選擇取決于特定任務(wù)和輸入數(shù)據(jù)的性質(zhì)。常見的相似性度量包括歐氏距離、余弦相似度或相關(guān)系數(shù)。相似性度量量化了embedding之間的距離或相關(guān)性，并提供了輸入樣本之間相似性的度量。
- 對(duì)比損失函數(shù)：為了訓(xùn)練Siamese網(wǎng)絡(luò)，采用了對(duì)比損失函數(shù)。對(duì)比損失函數(shù)鼓勵(lì)網(wǎng)絡(luò)為相似的輸入生成距離更近的embedding，而為不相似的輸入生成距離更遠(yuǎn)的embedding。當(dāng)相似對(duì)之間的距離超過一定閾值或不相似對(duì)之間的距離低于另一個(gè)閾值時(shí)，對(duì)比損失函數(shù)對(duì)模型進(jìn)行懲罰。對(duì)比損失函數(shù)的確切制定取決于所選的相似性度量和相似對(duì)與不相似對(duì)之間的期望邊際。
在訓(xùn)練過程中，Siamese網(wǎng)絡(luò)學(xué)會(huì)優(yōu)化其參數(shù)以最小化對(duì)比損失，并生成能夠有效捕捉輸入數(shù)據(jù)的相似性結(jié)構(gòu)的判別性embedding。
對(duì)比損失函數(shù)
對(duì)比損失是Siamese網(wǎng)絡(luò)中常用于學(xué)習(xí)輸入樣本對(duì)之間相似性或不相似性的損失函數(shù)。它旨在以這樣一種方式優(yōu)化網(wǎng)絡(luò)的參數(shù)，即相似的輸入具有在特征空間中更接近的embedding，而不相似的輸入則被推到更遠(yuǎn)的位置。通過最小化對(duì)比損失，網(wǎng)絡(luò)學(xué)會(huì)生成能夠有效捕捉輸入數(shù)據(jù)的相似性結(jié)構(gòu)的embedding。
為了詳細(xì)了解對(duì)比損失函數(shù)，讓我們將其分解為其關(guān)鍵組件和步驟：
- 輸入對(duì)：對(duì)比損失函數(shù)作用于輸入樣本對(duì)，其中每對(duì)包含一個(gè)相似或正例和一個(gè)不相似或負(fù)例。這些對(duì)通常在訓(xùn)練過程中生成，其中正例對(duì)代表相似實(shí)例，而負(fù)例對(duì)代表不相似實(shí)例。
- embedding：Siamese網(wǎng)絡(luò)通過共享網(wǎng)絡(luò)處理每個(gè)輸入樣本，為配對(duì)中的兩個(gè)樣本生成embedding向量。這些embedding是固定長(zhǎng)度的表示，捕捉輸入樣本的基本特征。
- 距離度量：使用距離度量，如歐氏距離或余弦相似度，來(lái)衡量生成的embedding之間的不相似性或相似性。距離度量的選擇取決于輸入數(shù)據(jù)的性質(zhì)和任務(wù)的具體要求。
- 對(duì)比損失計(jì)算：對(duì)比損失函數(shù)計(jì)算每對(duì)embedding的損失，鼓勵(lì)相似對(duì)具有更小的距離，而不相似對(duì)具有更大的距離。對(duì)比損失的一般公式如下：L = (1 — y) * D2 + y * max(0, m — D)
其中：
- ?L：對(duì)于一對(duì)的對(duì)比損失。
- ?D：embedding之間的距離或不相似性。
- ?y：標(biāo)簽，指示配對(duì)是否相似（相似為0，不相似為1）。
- m：定義不相似性閾值的邊際參數(shù)。
損失項(xiàng) `(1 — y) * D2` 對(duì)相似對(duì)進(jìn)行懲罰，如果它們的距離超過邊際（m），則鼓勵(lì)網(wǎng)絡(luò)減小它們的距離。項(xiàng) `y * max(0, m — D)2` 對(duì)不相似對(duì)進(jìn)行懲罰，如果它們的距離低于邊際，則推動(dòng)網(wǎng)絡(luò)增加它們的距離。
- 損失的匯總：為了獲得整個(gè)輸入對(duì)批次的整體對(duì)比損失，通常會(huì)對(duì)所有對(duì)之間的個(gè)體損失進(jìn)行平均或求和。匯總方法的選擇取決于特定的訓(xùn)練目標(biāo)和優(yōu)化策略。
通過通過梯度下降優(yōu)化方法（例如反向傳播和隨機(jī)梯度下降）最小化對(duì)比損失，Siamese網(wǎng)絡(luò)學(xué)會(huì)生成能夠有效捕捉輸入數(shù)據(jù)的相似性結(jié)構(gòu)的判別性embedding。對(duì)比損失函數(shù)在訓(xùn)練Siamese網(wǎng)絡(luò)中發(fā)揮著關(guān)鍵作用，使其能夠?qū)W習(xí)可用于各種任務(wù)，如圖像相似性、人臉驗(yàn)證和文本相似性的有意義表示。對(duì)比損失函數(shù)的具體制定和參數(shù)可以根據(jù)數(shù)據(jù)的特性和任務(wù)的要求進(jìn)行調(diào)整。
在 PyTorch 中的孿生神經(jīng)網(wǎng)絡(luò)
1. 數(shù)據(jù)集創(chuàng)建
我們使用的數(shù)據(jù)集來(lái)自來(lái)自 ：
http://vision.stanford.edu/aditya86/ImageNetDogs/?
def copy_files(source_folder,files_list,des):for file in files_list:source_file=os.path.join(source_folder,file)des_file=os.path.join(des,file)shutil.copy2(source_file,des_file)print(f"Copied {file} to {des}")return def move_files(source_folder,des):files_list=os.listdir(source_folder)for file in files_list:source_file=os.path.join(source_folder,file)des_file=os.path.join(des,file)shutil.move(source_file,des_file)print(f"Copied {file} to {des}")return def rename_file(file_path,new_name):directory=os.path.dirname(file_path)new_file_path=os.path.join(directory,new_name)os.rename(file_path,new_file_path)print(f"File renamed to {new_file_path}")returnfolder_path=r"C:\Users\sri.karan\Downloads\images1\Images\*"op_path_similar=r"C:\Users\sri.karan\Downloads\images1\Images\similar_all_images"tmp=r"C:\Users\sri.karan\Downloads\images1\Images\tmp"op_path_dissimilar=r"C:\Users\sri.karan\Downloads\images1\Images\dissimilar_all_images"folders_list=glob.glob(folder_path)folders_list=list(set(folders_list).difference(set(['C:\\Users\\sri.karan\\Downloads\\images1\\Images\\similar_all_images','C:\\Users\\sri.karan\\Downloads\\images1\\Images\\tmp','C:\\Users\\sri.karan\\Downloads\\images1\\Images\\dissimilar_all_images'])))l,g=0,0random.shuffle(folders_list)for i in glob.glob(folder_path):if i in ['C:\\Users\\sri.karan\\Downloads\\images1\\Images\\similar_all_images','C:\\Users\\sri.karan\\Downloads\\images1\\Images\\tmp','C:\\Users\\sri.karan\\Downloads\\images1\\Images\\dissimilar_all_images']:continuefile_name=i.split('\\')[-1].split("-")[1]picked_files=pick_random_files(i,6)copy_files(i,picked_files,tmp)for m in range(3):rename_file(os.path.join(tmp,picked_files[m*2]),"similar_"+str(g)+"_first.jpg")rename_file(os.path.join(tmp,picked_files[m*2+1]),"similar_"+str(g)+"_second.jpg")g+=1move_files(tmp,op_path_similar)choice_one,choice_two=random.choice(range(len(folders_list))),random.choice(range(len(folders_list)))picked_dissimilar_one=pick_random_files(folders_list[choice_one],3)picked_dissimilar_two=pick_random_files(folders_list[choice_two],3)copy_files(folders_list[choice_one],picked_dissimilar_one,tmp)copy_files(folders_list[choice_two],picked_dissimilar_two,tmp)picked_files_dissimilar=picked_dissimilar_one+picked_dissimilar_twofor m in range(3):rename_file(os.path.join(tmp,picked_files_dissimilar[m]),"dissimilar_"+str(l)+"_first.jpg")rename_file(os.path.join(tmp,picked_files_dissimilar[m+3]),"dissimilar_"+str(l)+"_second.jpg")l+=1move_files(tmp,op_path_dissimilar)我們挑選了3對(duì)相似圖像（狗品種）和3對(duì)不相似圖像（狗品種）來(lái)微調(diào)模型，為了使負(fù)樣本簡(jiǎn)單，對(duì)于給定的錨定圖像（狗品種），任何除地面實(shí)況狗品種以外的其他狗品種都被視為負(fù)標(biāo)簽。
注意：?“相似圖像” 意味著來(lái)自相同狗品種的圖像被視為正對(duì)，而“不相似圖像” 意味著來(lái)自不同狗品種的圖像被視為負(fù)對(duì)。
代碼解釋：
- 46行：從每個(gè)狗圖像文件夾中隨機(jī)挑選了6張圖像。
- 47行：選擇的圖像被移動(dòng)到一個(gè)名為 “tmp” 的文件夾中，并且由于它們來(lái)自同一狗品種文件夾，因此被重命名為 “similar_images”。
- 55行：完成所有這些后，它們被移動(dòng)到 “similar_all_images” 文件夾中。
- 56、57行：類似地，為了獲得不相似的圖像對(duì)，從兩個(gè)不同的狗品種文件夾中選擇了3張圖像。
- 然后重復(fù)上述流程，以獲得不相似的圖像對(duì)并將它們移動(dòng)到 “dissimilar_all_images” 文件夾中。
完成所有這些后，我們可以繼續(xù)創(chuàng)建數(shù)據(jù)集對(duì)象。
import torchimport torch.nn as nnimport torch.optim as optimfrom torch.utils.data import DataLoaderfrom PIL import Imageimport numpy as npimport randomfrom torch.utils.data import DataLoader, Datasetimport torchimport torch.nn as nnfrom torch import optimimport torch.nn.functional as Fclass ImagePairDataset(torch.utils.data.Dataset):def __init__(self, root_dir):self.root_dir = root_dirself.transform = T.Compose([# We first resize the input image to 256x256 and then we take center crop.transforms.Resize((256,256)), transforms.ToTensor()])self.image_pairs = self.load_image_pairs()def __len__(self):return len(self.image_pairs)def __getitem__(self, idx):image1_path, image2_path, label = self.image_pairs[idx]image1 = Image.open(image1_path).convert("RGB")image2 = Image.open(image2_path).convert("RGB")# Convert the tensor to a PIL image# image1 = functional.to_pil_image(image1)# image2 = functional.to_pil_image(image2)image1 = self.transform(image1)image2 = self.transform(image2)# image1 = torch.clamp(image1, 0, 1)# image2 = torch.clamp(image2, 0, 1)return image1, image2, labeldef load_image_pairs(self):image_pairs = []# Assume the directory structure is as follows:# root_dir# ├── similar# │ ├── similar_image1.jpg# │ ├── similar_image2.jpg# │ └── ...# └── dissimilar# ├── dissimilar_image1.jpg# ├── dissimilar_image2.jpg# └── ...similar_dir = os.path.join(self.root_dir, "similar_all_images")dissimilar_dir = os.path.join(self.root_dir, "dissimilar_all_images")# Load similar image pairs with label 1similar_images = os.listdir(similar_dir)for i in range(len(similar_images) // 2):image1_path = os.path.join(similar_dir, f"similar_{i}_first.jpg")image2_path = os.path.join(similar_dir, f"similar_{i}_second.jpg")image_pairs.append((image1_path, image2_path, 0))# Load dissimilar image pairs with label 0dissimilar_images = os.listdir(dissimilar_dir)for i in range(len(dissimilar_images) // 2):image1_path = os.path.join(dissimilar_dir, f"dissimilar_{i}_first.jpg")image2_path = os.path.join(dissimilar_dir, f"dissimilar_{i}_second.jpg")image_pairs.append((image1_path, image2_path, 1))return image_pairsdataset = ImagePairDataset(r"/home/niq/hcsr2001/data/image_similarity")train_size = int(0.8 * len(dataset))test_size = len(dataset) - train_sizetrain_dataset, test_dataset = torch.utils.data.random_split(dataset, [train_size, test_size])batch_size = 32train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)test_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)在上述代碼的第8到10行：對(duì)圖像進(jìn)行預(yù)處理，包括將圖像調(diào)整大小為256。我們使用批量大小為32，這取決于您的計(jì)算能力和 GPU。
#create the Siamese Neural Networkclass SiameseNetwork(nn.Module):def __init__(self):super(SiameseNetwork, self).__init__()# Setting up the Sequential of CNN Layers# self.cnn1 = nn.Sequential(# nn.Conv2d(3, 256, kernel_size=11,stride=4),# nn.ReLU(inplace=True),# nn.MaxPool2d(3, stride=2),# nn.Conv2d(256, 256, kernel_size=5, stride=1),# nn.ReLU(inplace=True),# nn.MaxPool2d(2, stride=2),# nn.Conv2d(256, 384, kernel_size=3,stride=1),# nn.ReLU(inplace=True)# )self.cnn1=nn.Conv2d(3, 256, kernel_size=11,stride=4)self.relu = nn.ReLU()self.maxpool1=nn.MaxPool2d(3, stride=2)self.cnn2=nn.Conv2d(256, 256, kernel_size=5,stride=1)self.maxpool2=nn.MaxPool2d(2, stride=2)self.cnn3=nn.Conv2d(256, 384, kernel_size=3,stride=1)self.fc1 =nn.Linear(46464, 1024)self.fc2=nn.Linear(1024, 256)self.fc3=nn.Linear(256, 1)# Setting up the Fully Connected Layers# self.fc1 = nn.Sequential(# nn.Linear(384, 1024),# nn.ReLU(inplace=True),# nn.Linear(1024, 32*46464),# nn.ReLU(inplace=True),# nn.Linear(32*46464,1)# )def forward_once(self, x):# This function will be called for both images# Its output is used to determine the similiarity# output = self.cnn1(x)# print(output.view(output.size()[0], -1).shape)# output = output.view(output.size()[0], -1)# output = self.fc1(output)# print(x.shape)output= self.cnn1(x)# print(output.shape)output=self.relu(output)# print(output.shape)output=self.maxpool1(output)# print(output.shape)output= self.cnn2(output)# print(output.shape)output=self.relu(output)# print(output.shape)output=self.maxpool2(output)# print(output.shape)output= self.cnn3(output)output=self.relu(output)# print(output.shape)output=output.view(output.size()[0], -1)# print(output.shape)output=self.fc1(output)# print(output.shape)output=self.fc2(output)# print(output.shape)output=self.fc3(output)return outputdef forward(self, input1, input2):# In this function we pass in both images and obtain both vectors# which are returnedoutput1 = self.forward_once(input1)output2 = self.forward_once(input2)return output1, output2
我們的網(wǎng)絡(luò)稱為 SiameseNetwork，我們可以看到它幾乎與標(biāo)準(zhǔn) CNN 相同。唯一可以注意到的區(qū)別是我們有兩個(gè)前向函數(shù)（forward_once 和 forward）。為什么呢？
我們提到通過相同網(wǎng)絡(luò)傳遞兩個(gè)圖像。forward_once 函數(shù)在 forward 函數(shù)中調(diào)用，它將一個(gè)圖像作為輸入傳遞到網(wǎng)絡(luò)。輸出存儲(chǔ)在 output1 中，而來(lái)自第二個(gè)圖像的輸出存儲(chǔ)在 output2 中，正如我們?cè)?forward 函數(shù)中看到的那樣。通過這種方式，我們?cè)O(shè)法輸入了兩個(gè)圖像并從我們的模型獲得了兩個(gè)輸出。
我們已經(jīng)看到了損失函數(shù)應(yīng)該是什么樣子，現(xiàn)在讓我們來(lái)編碼它。我們創(chuàng)建了一個(gè)名為 ContrastiveLoss 的類，與模型類一樣，我們將有一個(gè) forward 函數(shù)。
class ContrastiveLoss(torch.nn.Module):def __init__(self, margin=2.0):super(ContrastiveLoss, self).__init__()self.margin = margindef forward(self, output1, output2, label):# Calculate the euclidean distance and calculate the contrastive losseuclidean_distance = F.pairwise_distance(output1, output2, keepdim = True)loss_contrastive = torch.mean((1-label) * torch.pow(euclidean_distance, 2) +(label) * torch.pow(torch.clamp(self.margin - euclidean_distance, min=0.0), 2))return loss_contrastivenet = SiameseNetwork().cuda()criterion = ContrastiveLoss()optimizer = optim.Adam(net.parameters(), lr = 0.0005 )
按照頂部的流程圖，我們可以開始創(chuàng)建訓(xùn)練循環(huán)。我們迭代100次并提取兩個(gè)圖像以及標(biāo)簽。我們將梯度歸零，將兩個(gè)圖像傳遞到網(wǎng)絡(luò)中，網(wǎng)絡(luò)輸出兩個(gè)向量。然后，將兩個(gè)向量和標(biāo)簽饋送到我們定義的 criterion（損失函數(shù)）中。我們進(jìn)行反向傳播和優(yōu)化。出于一些可視化目的，并查看我們的模型在訓(xùn)練集上的性能，因此我們將每10批次打印一次損失。
counter = []loss_history = [] iteration_number= 0# Iterate throught the epochsfor epoch in range(100):# Iterate over batchesfor i, (img0, img1, label) in enumerate(train_loader, 0):# Send the images and labels to CUDAimg0, img1, label = img0.cuda(), img1.cuda(), label.cuda()# Zero the gradientsoptimizer.zero_grad()# Pass in the two images into the network and obtain two outputsoutput1, output2 = net(img0, img1)# Pass the outputs of the networks and label into the loss functionloss_contrastive = criterion(output1, output2, label)# Calculate the backpropagationloss_contrastive.backward()# Optimizeoptimizer.step()# Every 10 batches print out the lossif i % 10 == 0 :print(f"Epoch number {epoch}\n Current loss {loss_contrastive.item()}\n")iteration_number += 10counter.append(iteration_number)loss_history.append(loss_contrastive.item())show_plot(counter, loss_history)
我們現(xiàn)在可以分析結(jié)果。我們能看到的第一件事是損失從1.6左右開始，并以接近1的數(shù)字結(jié)束。看到模型的實(shí)際運(yùn)行情況將是有趣的?，F(xiàn)在是我們?cè)谀Ｐ椭皼]見過的圖像上測(cè)試我們的模型的部分。與之前一樣，我們使用我們的自定義數(shù)據(jù)集類創(chuàng)建了一個(gè) Siamese Network 數(shù)據(jù)集，但現(xiàn)在我們將其指向測(cè)試文件夾。
作為接下來(lái)的步驟，我們從第一批中提取第一張圖像，并迭代5次以提取接下來(lái)5批中的5張圖像，因?yàn)槲覀冊(cè)O(shè)置每批包含一張圖像。然后，使用 torch.cat() 水平組合兩個(gè)圖像，我們可以清楚地可視化哪個(gè)圖像與哪個(gè)圖像進(jìn)行了比較。
我們將兩個(gè)圖像傳入模型并獲得兩個(gè)向量，然后將這兩個(gè)向量傳入 F.pairwise_distance() 函數(shù)，這將計(jì)算兩個(gè)向量之間的歐氏距離。使用這個(gè)距離，我們可以作為衡量?jī)蓮埬樣卸嗖幌嗨频闹笜?biāo)。
test_loader_one = DataLoader(test_dataset, batch_size=1, shuffle=False)dataiter = iter(test_loader_one)x0, _, _ = next(dataiter)for i in range(5):# Iterate over 5 images and test them with the first image (x0)_, x1, label2 = next(dataiter)# Concatenate the two images togetherconcatenated = torch.cat((x0, x1), 0)output1, output2 = net(x0.cuda(), x1.cuda())euclidean_distance = F.pairwise_distance(output1, output2)imshow(torchvision.utils.make_grid(concatenated), f'Dissimilarity: {euclidean_distance.item():.2f}')view raweval.py hosted with ? by GitHub
總結(jié)
Siamese 網(wǎng)絡(luò)與對(duì)比損失結(jié)合，為學(xué)習(xí)圖像相似性提供了一個(gè)強(qiáng)大而有效的框架。通過對(duì)相似和不相似圖像進(jìn)行訓(xùn)練，這些網(wǎng)絡(luò)可以學(xué)會(huì)提取能夠捕捉基本視覺特征的判別性embedding。對(duì)比損失函數(shù)通過優(yōu)化embedding空間進(jìn)一步增強(qiáng)
了模型準(zhǔn)確測(cè)量圖像相似性的能力。隨著深度學(xué)習(xí)和計(jì)算機(jī)視覺的進(jìn)步，Siamese 網(wǎng)絡(luò)在各個(gè)領(lǐng)域都有著巨大的潛力，包括圖像搜索、人臉驗(yàn)證和推薦系統(tǒng)。通過利用這些技術(shù)，我們可以為基于內(nèi)容的圖像檢索、視覺理解以及視覺領(lǐng)域的智能決策開啟令人興奮的可能性。
The above is the detailed content of Exploring Siamese networks using contrastive loss for image similarity comparison. For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undress AI Tool
Undress images for free
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Bytedance Cutting launches SVIP super membership: 499 yuan for continuous annual subscription, providing a variety of AI functions
Jun 28, 2024 am 03:51 AM
This site reported on June 27 that Jianying is a video editing software developed by FaceMeng Technology, a subsidiary of ByteDance. It relies on the Douyin platform and basically produces short video content for users of the platform. It is compatible with iOS, Android, and Windows. , MacOS and other operating systems. Jianying officially announced the upgrade of its membership system and launched a new SVIP, which includes a variety of AI black technologies, such as intelligent translation, intelligent highlighting, intelligent packaging, digital human synthesis, etc. In terms of price, the monthly fee for clipping SVIP is 79 yuan, the annual fee is 599 yuan (note on this site: equivalent to 49.9 yuan per month), the continuous monthly subscription is 59 yuan per month, and the continuous annual subscription is 499 yuan per year (equivalent to 41.6 yuan per month) . In addition, the cut official also stated that in order to improve the user experience, those who have subscribed to the original VIP
Context-augmented AI coding assistant using Rag and Sem-Rag
Jun 10, 2024 am 11:08 AM
Improve developer productivity, efficiency, and accuracy by incorporating retrieval-enhanced generation and semantic memory into AI coding assistants. Translated from EnhancingAICodingAssistantswithContextUsingRAGandSEM-RAG, author JanakiramMSV. While basic AI programming assistants are naturally helpful, they often fail to provide the most relevant and correct code suggestions because they rely on a general understanding of the software language and the most common patterns of writing software. The code generated by these coding assistants is suitable for solving the problems they are responsible for solving, but often does not conform to the coding standards, conventions and styles of the individual teams. This often results in suggestions that need to be modified or refined in order for the code to be accepted into the application
Seven Cool GenAI & LLM Technical Interview Questions
Jun 07, 2024 am 10:06 AM
To learn more about AIGC, please visit: 51CTOAI.x Community https://www.51cto.com/aigc/Translator|Jingyan Reviewer|Chonglou is different from the traditional question bank that can be seen everywhere on the Internet. These questions It requires thinking outside the box. Large Language Models (LLMs) are increasingly important in the fields of data science, generative artificial intelligence (GenAI), and artificial intelligence. These complex algorithms enhance human skills and drive efficiency and innovation in many industries, becoming the key for companies to remain competitive. LLM has a wide range of applications. It can be used in fields such as natural language processing, text generation, speech recognition and recommendation systems. By learning from large amounts of data, LLM is able to generate text
Can fine-tuning really allow LLM to learn new things: introducing new knowledge may make the model produce more hallucinations
Jun 11, 2024 pm 03:57 PM
Large Language Models (LLMs) are trained on huge text databases, where they acquire large amounts of real-world knowledge. This knowledge is embedded into their parameters and can then be used when needed. The knowledge of these models is "reified" at the end of training. At the end of pre-training, the model actually stops learning. Align or fine-tune the model to learn how to leverage this knowledge and respond more naturally to user questions. But sometimes model knowledge is not enough, and although the model can access external content through RAG, it is considered beneficial to adapt the model to new domains through fine-tuning. This fine-tuning is performed using input from human annotators or other LLM creations, where the model encounters additional real-world knowledge and integrates it
Five schools of machine learning you don't know about
Jun 05, 2024 pm 08:51 PM
Machine learning is an important branch of artificial intelligence that gives computers the ability to learn from data and improve their capabilities without being explicitly programmed. Machine learning has a wide range of applications in various fields, from image recognition and natural language processing to recommendation systems and fraud detection, and it is changing the way we live. There are many different methods and theories in the field of machine learning, among which the five most influential methods are called the "Five Schools of Machine Learning". The five major schools are the symbolic school, the connectionist school, the evolutionary school, the Bayesian school and the analogy school. 1. Symbolism, also known as symbolism, emphasizes the use of symbols for logical reasoning and expression of knowledge. This school of thought believes that learning is a process of reverse deduction, through existing
To provide a new scientific and complex question answering benchmark and evaluation system for large models, UNSW, Argonne, University of Chicago and other institutions jointly launched the SciQAG framework
Jul 25, 2024 am 06:42 AM
Editor |ScienceAI Question Answering (QA) data set plays a vital role in promoting natural language processing (NLP) research. High-quality QA data sets can not only be used to fine-tune models, but also effectively evaluate the capabilities of large language models (LLM), especially the ability to understand and reason about scientific knowledge. Although there are currently many scientific QA data sets covering medicine, chemistry, biology and other fields, these data sets still have some shortcomings. First, the data form is relatively simple, most of which are multiple-choice questions. They are easy to evaluate, but limit the model's answer selection range and cannot fully test the model's ability to answer scientific questions. In contrast, open-ended Q&A
A new era of VSCode front-end development: 12 highly recommended AI code assistants
Jun 11, 2024 pm 07:47 PM
In the world of front-end development, VSCode has become the tool of choice for countless developers with its powerful functions and rich plug-in ecosystem. In recent years, with the rapid development of artificial intelligence technology, AI code assistants on VSCode have sprung up, greatly improving developers' coding efficiency. AI code assistants on VSCode have sprung up like mushrooms after a rain, greatly improving developers' coding efficiency. It uses artificial intelligence technology to intelligently analyze code and provide precise code completion, automatic error correction, grammar checking and other functions, which greatly reduces developers' errors and tedious manual work during the coding process. Today, I will recommend 12 VSCode front-end development AI code assistants to help you in your programming journey.
SK Hynix will display new AI-related products on August 6: 12-layer HBM3E, 321-high NAND, etc.
Aug 01, 2024 pm 09:40 PM
According to news from this site on August 1, SK Hynix released a blog post today (August 1), announcing that it will attend the Global Semiconductor Memory Summit FMS2024 to be held in Santa Clara, California, USA from August 6 to 8, showcasing many new technologies. generation product. Introduction to the Future Memory and Storage Summit (FutureMemoryandStorage), formerly the Flash Memory Summit (FlashMemorySummit) mainly for NAND suppliers, in the context of increasing attention to artificial intelligence technology, this year was renamed the Future Memory and Storage Summit (FutureMemoryandStorage) to invite DRAM and storage vendors and many more players. New product SK hynix launched last year
