The use of head-mounted augmented reality (AR) for surgeries has grown rapidly in recent years. AR aids in intraoperative surgical navigation through overlaying 3D holographic reconstructions of medical data. However, performing AR surgeries on complex areas poses challenges in terms of accuracy and speed. This study explores the feasibility of an AR guidance system for resections of positive tumor margins at the head and neck region. We present an intraoperative solution that enables surgeons to access holographic reconstructions of resected cadaver tissues. The solution involves using 3D scanner to capture detailed scans of the resected tissue, which are uploaded into our software. It then converts these scans into holograms that are viewable through a head-mounted AR display. Surgeons navigates the tumor site by re-aligning these holograms with cadavers using gestures or voice commands. This workflow runs concurrently with frozen section analysis. On average, we achieve an uploading time of 2.98 min, visualization time of 1.05 min and re-alignment time of 4.39 min, within the 20 - 30 min window for frozen section analysis. We achieve a mean re-alignment error of 3.1 mm. Our software provides a foundation for new product development in using AR to navigate complex anatomy in surgery.

Kidney stones require surgical removal when they grow too large to be broken up externally or to pass on their own. Upper tract urothelial carcinoma are also sometimes treated endoscopically in a similar procedure. These surgeries are difficult, particularly for trainees who often miss tumors, stones or stone fragments, requiring re-operation. One cause of difficulty is the high cognitive strain surgeons experience in creating accurate mental models during the endoscopic operation. Furthermore, there are no patient-specific simulators to facilitate training or standardized visualization tools for ureteroscopy despite its high prevalence. We propose ASSIST-U, a system to automatically create realistic ureteroscopy images and videos solely using preoperative CT images to address these unmet needs. We train a 3D UNet model to automatically segment CT images and construct 3D surfaces. These surfaces are then skeletonized for rendering and camera position tracking. Finally, we train a style transfer model using Contrastive Unpaired Translation (CUT) to synthesize realistic ureteroscopy images. Cross validation on the UNet model achieved a Dice score of 0.853 $\pm$ 0.084 for the CT segmentation step. CUT style transfer produced visually plausible images; the Kernel Inception Distance to real ureteroscopy images was reduced from 0.198 (rendered) to 0.089 (synthesized). We also qualitatively demonstrate the entire pipeline from CT to synthesized ureteroscopy. The proposed ASSIST-U system shows promise for aiding surgeons in visualization of kidney ureteroscopy.

The integration of Augmented Reality (AR) into daily surgical practice is withheld by the correct registration of pre-operative data. This includes intelligent 3D model superposition whilst simultaneously handling real and virtual occlusions caused by the AR overlay. Occlusions can negatively impact surgical safety and as such deteriorate rather than improve surgical care. Robotic surgery is particularly suited to tackle these integration challenges in a stepwise approach as the robotic console allows for different inputs to be displayed in parallel to the surgeon. Nevertheless, real-time de-occlusion requires extensive computational resources which further complicates clinical integration. This work tackles the problem of instrument occlusion and presents, to our best knowledge, the first-in-human on edge deployment of a real-time binary segmentation pipeline during three robot-assisted surgeries: partial nephrectomy, migrated endovascular stent removal and liver metastasectomy. To this end, a state-of-the-art real-time segmentation and 3D model pipeline was implemented and presented to the surgeon during live surgery. The pipeline allows real-time binary segmentation of 37 non-organic surgical items, which are never occluded during AR. The application features real-time manual 3D model manipulation for correct soft tissue alignment. The proposed pipeline can contribute towards surgical safety, ergonomics and acceptance of AR in minimally invasive surgery.

Efficient communication and collaboration are essential in the operating room for successful and safe surgery. While many technologies are improving various aspects of surgery, communication between attending surgeons, residents, and surgical teams is still limited to verbal interactions that are prone to misunderstandings. Novel modes of communication can increase speed and accuracy, and transform operating rooms. We present a mixed reality (MR) based gaze sharing application on Microsoft HoloLens 2 headset that can help expert surgeons indicate specific regions, communicate with decreased verbal effort, and guide novices throughout an operation. We test the utility of the application with a user study of endoscopic kidney stone localization completed by urology expert and novice surgeons. We observe improvement in the NASA Task Load Index surveys (up to 25.23%), in the success rate of the task (6.98% increase in localized stone percentage), and in gaze analyses (up to 31.99%). The proposed application shows promise in both operating room applications and surgical training tasks.

Endoscopic renal surgeries have high re-operation rates, particularly for lower volume surgeons. Due to the limited field and depth of view of current endoscopes, mentally mapping preoperative computed tomography (CT) images of patient anatomy to the surgical field is challenging. The inability to completely navigate the intrarenal collecting system leads to missed kidney stones and tumors, subsequently raising recurrence rates. We propose a guidance system to estimate the endoscope positions within the CT to reduce re-operation rates. We use a Structure from Motion algorithm to reconstruct the kidney collecting system from the endoscope videos. In addition, we segment the kidney collecting system from CT scans using 3D U-Net to create a 3D model. We can then register the two collecting system representations to provide information on the relative endoscope position. We demonstrate correct reconstruction and localization of intrarenal anatomy and endoscope position. Furthermore, we create a 3D map supported by the RGB endoscope images to reduce the burden of mental mapping during surgery. The proposed reconstruction pipeline has been validated for guidance. It can reduce the mental burden for surgeons and is a step towards our long-term goal of reducing re-operation rates in kidney stone surgery.

This work present a proof-of-concept of a robotic-driven intra-operative scanner designed for knee cartilage lesion repair, part of a system for direct in vivo bioprinting. The proposed system is based on a photogrammetric pipeline, which reconstructs the cartilage and lesion surfaces from sets of photographs acquired by a robotic-handled endoscope, and produces 3D grafts for further printing path planning. A validation on a synthetic phantom is presented, showing that —despite the cartilage smooth and featureless surface— the current prototype can accurately reconstruct osteochondral lesions and their surroundings with mean error values of 0.199 ± 0.096 mm but with noticeable concentration on areas with poor lighting or low photographic coverage. The system can also accurately generate grafts for bioprinting, although with a slight tendency to underestimate the actual lesion sizes, producing grafts with coverage errors of -12.2 ± 3.7, -7.9 ± 4.9 and -15.2 ± 3.4 % for the medio-lateral, antero-posterior and craneo-caudal directions respectively. Improvements in lighting and acquisition for enhancing reconstruction accuracy are planned as future work, as well as integration into a complete bioprinting pipeline and validation with ex vivo phantoms.

Hand pose estimation based on a single RGB image has low accuracy due to the complexity of the pose, local self-similarity of finger features, and occlusion. A multiscale feature fusion network (MS-FF) for monocular vision gesture pose estimation is proposed to address this problem. The network can take full advantage of different channel information to enhance important gesture information, and it can simultaneously extract features from feature maps of different resolutions to obtain as much detailed feature information and deep semantic information as possible. The feature maps are merged to obtain the hand pose results. The InterHand2.6M dataset and Rendered Handpose Dataset (RHD) are used to train the MS-FF. Compared with the other methods (which can estimate interacting hand poses from a single RGB image), the MS-FF obtains the smallest average error of hand joints on RHD, verifying its effectiveness.

This letter proposes a polarization microwave correlation imaging method based on the orthogonal complement space. It utilizes the orthogonal complement space of the HV antenna radiation field to cross-multiply the echo information, enabling simultaneous correlation imaging and instantaneous polarization measurement of the target. Currently, there is a significant research gap in polarization-driven microwave correlation imaging methods, and the existing relevant studies focus on enhancing the randomness of the radiation field using polarized antenna elements, without incorporating polarization information into the imaging process. Through simulation analysis, this method further improves the quality of microwave correlation imaging and its ability to resist interference. Moreover, under low time-frequency products, the peak sidelobe level (PSL) and isolation (I) of this method are approximately 3.5 dB and 12.5 dB higher, respectively, than those of traditional instantaneous polarization measurement (TIPM)methods.

The existing ripeness detection algorithm for strawberries suffers from low detection accuracy and high detection error rate. Considering these problems, we propose an improvement method based on YOLOv5, which firstly reconfigures the feature extraction network by replacing ordinary convolution with hybrid depth deformable convolution. In the second step, a double cooperative attention mechanism is constructed to improve the representation of strawberry features in complex environments. Finally, cross-scale feature fusion is proposed to fully integrate the multiscale target features. The method was tested on the strawberry ripeness dataset, the mAP reached 95.6 percentage points, the FPS reached 76, and the model size was 7.44M. The mAP and FPS are 8.4 and 1.3 percentage points higher respectively than the baseline network. The model size is reduced by 6.28M. This method is superior to many state-of-the-art algorithms in terms of detection speed and accuracy. The system can accurately identify the ripeness of strawberries in complex environments, which could provide technical support for automated picking robots.

Considering light absorbing and scattering problems in connection with wavelength can decrease the visibility, contrast and color distortion of images, we propose a new type of convolutional neural network with two training phases. Firstly, the coordinate attention module is integrated into the residual block of the residual group in the backbone network, which is used to strengthen the feature extraction capability of the network. Secondly, since the unrealistic image colors may degrade the image details, an unsupervised method that combines the physical prior knowledge and the real underwater images is proposed to finetune the backbone network. Furthermore, a model protection mechanism is designed to guarantee the successful execution of the training. The experimental results indicate the proposed model can effectively optimize the contrast, color and image quality of the underwater image. Compared with relevant algorithms, our UCIQE and NIQE are respectively 0.525 and 4.149, which further verifies the superiority of the proposed model.

Secure and reliable electricity supply is a prerequisite for the development of smart cities, and the trustworthy and efficient transmission of electrical data is the foundation for the safe and stable operation of the power grid. This paper introduces a real-time data transmission blockchain technique based on parallel proof of work algorithm. The new block generation progress of proposed blockchain is divided into five subroutines: hash pointer computation, real-time data pudding, signature value iteration, interruption, block header assembly. The real-time data pudding and signature value iteration are parallel processed, which brings the effect of decreasing energy loss of blockchain system, and upgrades the speed of new block generation and the bandwidth of data storing on blockchain. Computer simulation shows the proposed strategy can be effectively applied in real-time electrical data transmission application, raising the data transmission reliability with no harm to real-time data transfer function. This strategy provides a solution to guarantee data transmission safety in the digital conversion of power grid.

As renewable power generation increases in distribution networks, the real-time power balance is becoming a tough challenge. Unlike simple peak-load shedding or demand turn-down scenarios, generation following requires persistent and precise control due to the temporal response performance of controlled resources. This motivates a comprehensive control design considering the temporal response limitations and execution performance of ACCs when providing such services. Accordingly, this paper proposes a self-constraint MPC that properly allocates the generation following task among different ACCs, consisting of three main parts: response rehearsal, distributed consistency-based power allocation, and real-time task execution. Specifically, the rehearsal knowledge of ACCs is evaluated by introducing model predictive control to track power signals with different values and thus obtain prior factors, including the upward/downward limits and control cost function. On this basis, the coherence of the incremental response costs of different clusters is achieved by containing the prior factors to model the constraints and cost functions. Once the optimised following signals are obtained, a real-time model predictive controller for generation following task execution is employed. Simulations are conducted to verify the feasibility and effectiveness of the proposed method.

In this study, an eye blinking re-identification system was proposed. A fast local binary pattern was used for feature extraction because its grayscale invariance and rotational invariance allow for the effective acquisition of feature information even in the presence of noise. Finally, a recurrent neural network and long short-term memory were used for model training. The results indicated that, compared with the model trained using static data, the models based on dynamic features were less affected by environmental noise in terms of accuracy. In addition, the model trained using the recurrent neural network was highly effective in identifying unenrolled users and achieved high overall accuracy.

This study proposed a matched field source localization method based on tensor decomposition. By considering the advantages of tensors in multidimensional data processing, a three-dimensional tensor signal model of space-time-frequency is constructed, and the signal subspace is estimated using high-order singular value decomposition (HOSVD). The source position is estimated by matching the measured data tensor signal subspace with the replica field tensor signal subspace. The S5 event data of SWellEx-96 is processed by the proposed tensor-based matched-field processing (TMFP). The comparison with the results of conventional matched field processing (MFP) shows that TMFP has a better suppression effect on ambient noise under low SNR and better source localization performance.

A novel stripline diplexer design using frequency dependent couplings to achieve multiple transmission zeros is developed in this paper. The transmission zeros generated by the frequency dependent couplings are flexible and controllable, on the basis of the existing cross-coupled, more transmission zeros are introduced to improve the frequency selection characteristics. Based on this characteristic, we designed a 2.6G Hz diplexer, its transmitting channel filter is 5 order with 4 transmission zeros, and the receiving channel filter is 4 order with 5 transmission zeros. We fabricated and measured it, the synthesis results, simulation results, and the tested results are well matched with each other, which will provide more flexibility in the design of diplexers for wireless communication system.

In this work, we report the first demonstration of an ultraviolet light-emitting diode (LED) with boron-containing multiple quantum wells. Electroluminescence emission from the BAlGaN LED was observed at 350 nm, with higher intensity compared to the AlGaN reference LED. A higher operating voltage compared to the reference LED was also observed which may be attributable to a nanomasking behaviour of boron in (Al)GaN alloys.

Consider general minimum variance distortionless response (MVDR) robust adaptive beamforming problems based on the optimal estimation for both the desired signal steering vector and the interference-plus-noise covariance (INC) matrix. The optimal robust adaptive beamformer design problem is an array output power maximization problem, subject to three constraints on the steering vector, namely, a (convex or nonconvex) quadratic constraint ensuring that the direction-of-arrival (DOA) of the desired signal is separated from the DOA region of all linear combinations of the interference steering vectors, a double-sided norm constraint, and a similarity constraint; as well as a ball constraint on the INC matrix, which is centered at a given data sample covariance matrix. To tackle the nonconvex problem, a new tightened semidefinite relaxation (SDR) approach is proposed to output a globally optimal solution; otherwise, a sequential convex approximation (SCA) method is established to return a locally optimal solution. The simulation results show that the MVDR robust adaptive beamformers based on the optimal estimation for the steering vector and the INC matrix have better performance (in terms of, e.g., the array output signal-to-interference-plus-noise ratio) than the existing MVDR robust adaptive beamformers by the steering vector estimation only.

This letter presents a sub-6 GHz wideband low noise amplifier (LNA) based on double L-type load network and negative feedback technique. Using the cascode structure combined with the above techniques, a single-stage wideband LNA with high gain and low noise figure (NF) can be realized. Fabricated in 110-nm SOI CMOS technology, the proposed LNA achieves a maximum power gain of 15.2 dB, noise figure (NF) of 1.0–1.56 dB. The 3-dB bandwidth ranges from 3.05–4.55 GHz. The minimum power input at 1dB compression point (IP1dB) is -17.1 dBm. The LNA core area is 0.18 mm2 and dissipates a total power of 11.5 mW from 1.4 V power supply.