How to successfully open the camera and display the detection box on the html web page developed by flask and yolov5?

When developing html web pages using flask framework and yolov5, it is common to open the camera and perform real-time detection. However, sometimes, there are problems that the detection box cannot be displayed successfully. Below we will analyze the problem step by step and provide solutions.

First, let’s take a look at the front-end code:

 
    <div class="row" style="padding:3%;">
        <div class="col-lg-6">
            <h5>Input data:</h5>
            <div>
                <video id="video" autoplay></video>
            </div>
        </div>
        <div class="col-lg-6">
            <h5>Output result:</h5>
            <div class="class=" custom-file-container__image-preview>
                <img  src="/static/imghw/default1.png" data-src="#" class="lazy" id="res" alt="How to solve the problem that the camera cannot display detection boxes on HTML pages developed by Flask and YOLOv5?" >
            </div>
        </div>
    </div>

    <input type="button" onclick="start()" value="start recognition">
    <input type="button" onclick="stop()" value="pause recognition">

    <script>
    function start() {
         navigator.mediadevices.getusermedia({ video: true })
      .then(function (stream) {
        var video = document.queryselector(&#39;video&#39;);
        video.srcobject = stream;
        var canvas = document.createelement(&#39;canvas&#39;);
        var ctx = canvas.getcontext(&#39;2d&#39;);

        setinterval(function () {
        var videowidth = video.videowidth;
          var videoheight = video.videoheight;
          canvas.width = videowidth;
          canvas.height = videoheight;
          ctx.drawimage(video, 0, 0, videowidth, videoheight);
          var imagedata = canvas.todataurl(&#39;image/png&#39;,1); // Compress the image// Send data to the backend $.ajax({
            type: &#39;post&#39;,
            url: &#39;/image_data&#39;,
            data: {
            id:$("#uid").val(),
            image_data: imagedata
            },
            success: function (response) {
              console.log(response);
            }
          });
        }, 1000 / 30); // 30 frames per second})
        $("#res").attr("src", "/img_feed?id=" $("#uid").val())
      .catch(function (error) {
        console.error(error);
      });
    }
</script>

Next is the backend code:

 # Video streaming def gen(path):
    cap = cv2.VideoCapture(path)
    While cap.isOpened():
        try:
            # Record the start time start_time = time.time()
            # Get the screen success, frame = cap.read()
            if success:
                im, label, c = d.detect(frame)
                ret, jpeg = cv2.imencode('.png', im)
                if ret:
                    frame = jpeg.tobytes()
                    # Calculate the processing time elapsed_time = time.time() - start_time
                    print(f"Frame processing time: {elapsed_time:.3f} seconds")
                    yield (b'--frame\r\n'
                           b'Content-Type: image/jpeg\r\n\r\n' frame b'\r\n\r\n\r\n')
                else:
                    break
            else:
                break
        except Exception as e:
            print(e)
            Continue continue
    cap.release()
    cv2.VideoCapture(path)

# Video streaming result @app.route('/video_feed')
def video_feed():
    f = request.args.get("f")
    print(f'upload/{f}')
    return Response(gen(f'upload/{f}'),
                    mimetype='multipart/x-mixed-replace; boundary=frame')

# Front-end push stream @app.route('/image_data', methods=["POST"])
def image_data():
    image_data = request.form.get('image_data')
    id = request.form.get('id')
    image_data = io.BytesIO(base64.b64decode(image_data.split(',')[1]))
    img = Image.open(image_data)
    # Process the image, such as compression, filtering, etc. Output = io.BytesIO()
    img.save(output, format='PNG', quality=85)
    output.seek(0)
    # Save the processed image to the server img.save(f'upload/temp{id}.png')
    with open(f'upload/temp{id}.png', 'wb') as f:
        f.write(output.read())
    return "ok"

User feedback said that the detection box cannot be displayed when the camera is turned on and hopes to correctly identify the confidence of the image.

The key to the problem lies in the following points:

1. Camera path issues :
   In cv2.videocapture(path), the path parameter needs to be set correctly. It can be the following situations:

   • Local laptop camera: Fill in the number 0
   • RTSP address of IP camera
   • Local absolute path files (such as mp4, jpeg, etc.)

   But in your code, what is f passed through the interface by gen(f'upload/{f}')? This needs to be clear.

2. Error message :
   No specific error message is provided, which makes the problem diagnosis more difficult. If there is any error message, it is recommended to provide it for further analysis.
3. Interface calling problem :
   The /video_feed interface you mentioned is not called in the front-end code. It is necessary to ensure that the front-end calls this interface correctly to display the detection results.

To solve this problem, we can take the following steps:

• Check the camera path : Make sure that the path parameter in cv2.videocapture(path) is set correctly. If it is a local camera, try using 0. If it is a file path, make sure to use an absolute or full path.
• Front-end calls back-end interface : In the front-end start() function, the /video_feed interface should be called to obtain the detection results and displayed in the img tag. For example, you can add a call to /video_feed within the setinterval function and update the src property of the img tag.
• View error message : If there is any error message, carefully review and analyze the cause of the error, which may be permission problems, path errors or other configuration problems.

Through the above steps, the problem of not being able to display the detection box when the camera is turned on should be solved and the confidence of the image should be correctly identified.

The above is the detailed content of How to solve the problem that the camera cannot display detection boxes on HTML pages developed by Flask and YOLOv5?. For more information, please follow other related articles on the PHP Chinese website!