Sparsh Mittal

Approximate computing offers significant gains in efficiency at the cost of minor errors. In this paper, we show that since approximate computing legitimizes controlled imprecision, this very relaxation can be exploited by an adversary to insert Trojans into approximate circuits. Since the minor errors introduced by the Trojan may be indistinguishable from those introduced by approximate computing, these Trojans can easily evade detection, yet they can severely degrade the end application's "quality of result" (QoR). By contrast, the conventional exact computing paradigm does not tolerate errors; hence, any inserted Trojan can be easily detected. Thus, we show that approximate circuits are more vulnerable to attacks, and this may nullify their efficiency advantages. We demonstrate our ideas through the two most foundational circuits, approximate adders and multipliers. We categorize the existing approximate adders and multipliers into broad families from the perspective of Trojan insertion strategies that an adversary might employ. We present a generalized framework to identify the suitable hardware Trojan insertion and masking sites within each family of approximate adders and multipliers. We also discuss the implications of these threats for a real-life application. Our work strongly emphasizes the need for better security measures and provides insights that will guide the development of robust digital systems capable of balancing the intricacies of approximation and security.

Early diagnosis plays a pivotal role in effectively treating numerous diseases, especially in healthcare scenarios where prompt and accurate diagnoses are essential. Contrastive learning (CL) has emerged as a promising approach for medical tasks, offering advantages over traditional supervised learning methods. However, in healthcare, patient metadata contains valuable clinical information that can enhance representations, yet existing CL methods often overlook this data. In this study, we propose an novel approach that leverages both clinical information and imaging data in contrastive learning to enhance model generalization and interpretability. Furthermore, existing contrastive methods may be prone to sampling bias, which can lead to the model capturing spurious relationships and exhibiting unequal performance across protected subgroups frequently encountered in medical settings. To address these limitations, we introduce Patient-aware Contrastive Learning (PaCL), featuring an inter-class separability objective (IeSO) and an intra-class diversity objective (IaDO). IeSO harnesses rich clinical information to refine samples, while IaDO ensures the necessary diversity among samples to prevent class collapse. We demonstrate the effectiveness of PaCL both theoretically through causal refinements and empirically across six real-world medical imaging tasks spanning three imaging modalities: ophthalmology, radiology, and dermatology. Notably, PaCL outperforms previous techniques across all six tasks

Recent years have seen a phenomenal rise in the performance and applications of transformer neural networks. The family of transformer networks, including Bidirectional Encoder Representations from Transformer (BERT), Generative Pretrained Transformer (GPT) and Vision Transformer (ViT), have shown their effectiveness across Natural Language Processing (NLP) and Computer Vision (CV) domains. Transformerbased networks such as ChatGPT have impacted the lives of common men. However, the quest for high predictive performance has led to an exponential increase in transformers' memory and compute footprint. Researchers have proposed techniques to optimize transformer inference at all levels of abstraction. This paper presents a comprehensive survey of techniques for optimizing the inference phase of transformer networks. We survey techniques such as knowledge distillation, pruning, quantization, neural architecture search and lightweight network design at the algorithmic level. We further review hardwarelevel optimization techniques and the design of novel hardware accelerators for transformers. We summarize the quantitative results on the number of parameters/FLOPs and the accuracy of several models/techniques to showcase the tradeoff exercised by them. We also outline future directions in this rapidly evolving field of research. We believe that this survey will educate both novice and seasoned researchers and also spark a plethora of research efforts in this field.

Approximate computing (AC) techniques provide overall performance gains in terms of power and energy savings at the cost of minor loss in application accuracy. For this reason, AC has emerged as a viable method for efficiently supporting several compute-intensive applications, e.g., machine learning, deep learning, and image processing, that can tolerate bounded errors in computations. However, most prior techniques do not consider the possibility of soft errors or malicious bit-flips in AC systems. These errors may interact with approximation-introduced errors in unforeseen ways, leading to disastrous consequences, such as the failure of computing systems. A recent research effort, FTApprox (DATE'21) proposes an error-resilient approximate data format. FTApprox stores two blocks, starting from the one containing the most significant valid (MSV) bit. It also stores location of the MSV block and protects them using error-correcting bits (ECBs). However, FTApprox has crucial limitations such as lack of flexibility, redundantly storing zeros in the MSV, etc. In this paper, we propose a novel storage format named Versatile Approximate Data Format (VADF) for storing approximate integer numbers while providing resilience to soft errors. VADF prescribes rules for storing, for example, a 32-bit number in either 8-bit, 12-bit or 16-bit numbers. VADF identifies the MSV bit and stores a certain number of bits following the MSV bit. It also stores the location of the MSV bit and protects it by ECBs. VADF does not explicitly store the MSB bit itself and this prevents VADF from accruing significant errors. VADF incurs lower error than both truncation methodologies and FTApprox. We further evaluate five image-processing and machine-learning applications and confirm that VADF provides higher application quality than FTApprox in the presence and absence of soft errors. Finally, VADF allows the use of narrow arithmetic units. For example, instead of using a 32-bit multiplier/adder, one can first use VADF (or FTApprox) to compress the data and then use a 8-bit multiplier/adder. Through this approach, VADF facilitates 95.97% and 79.3% energy savings in multiplication and addition, respectively. However, the subsequent reconversion of the 8-bit output data to 32-bit data using Inv-VADF(16,3,32) diminishes the energy savings by 9.6% for addition and 0.56% for multiplication operation, respectively. The code is available at https://github.com/CandleLabAI/VADF-ApproximateDataFormat-TECS.

PCB component classification and segmentation can be helpful for PCB waste recycling. However, the variance in shapes and sizes of PCB components presents crucial challenges. We propose PCBSegClassNet, a novel deep neural network for PCB component classification and segmentation. The network uses a two-branch design that captures the global context in one branch and spatial features in the other. The fusion of two branches allows the effective segmentation of components of various sizes and shapes. We reinterpret the skip connections as a learning module to learn features efficiently. We propose a texture enhancement module that utilizes texture information and spatial features to obtain precise boundaries of components. We introduce a loss function that combines DICE, IoU, and SSIM loss functions to guide the training process for precise pixel-level, patch-level, and map-level segmentation. Our network outperforms all previous state-of-the-art networks on both segmentation and classification tasks. For example, it achieves a DICE score of 96.3% and IoU score of 92.7% on the FPIC dataset. From the FPIC dataset, we crop the images of 25 component classes and release these 19158 images in open-source as the "FPIC-Component dataset". On this dataset, our network achieves a classification accuracy of 95.2%. Our model is much more lightweight than previous networks and achieves a segmentation throughput of 122 frame-per-second on a single GPU. We also showcase its ability to count the number of each component on a PCB.

Wildfires can cause significant damage to forests and endanger wildlife. Detecting these forest fires at the initial stages helps the authorities in preventing them from spreading further. In this paper, we first propose a novel technique, termed CIELAB-color technique, which detects fire based on the color of the fire in CIELAB color space. We train state-of-art CNNs to detect fire. Since deep learning (CNNs) and image processing have complementary strengths, we combine their strengths to propose an ensemble architecture. It uses two CNNs and the CIELAB-color technique and then performs majority voting to decide the final fire/no-fire prediction output. We finally propose a chain-of-classifiers technique which first tests an image using the CIELAB-color technique. If an image is flagged as no-fire, then it further checks the image using a CNN. This technique has lower model size than ensemble technique. On FLAME dataset, the ensemble technique provides 93.32% accuracy, outperforming both previous works (88.01% accuracy) and individually using either CNNs or CIELAB-color technique. The source code can be obtained from https://github.com/CandleLabAI/FireDetection.

Dehazing refers to removing the haze and restoring the details from hazy images. In this paper, we propose ClarifyNet, a novel, end-to-end trainable, convolutional neural network architecture for single image dehazing. We note that a high-pass filter detects sharp edges, texture, and other fine details in the image, whereas a low-pass filter detects color and contrast information. Based on this observation, our key idea is to train ClarifyNet on ground-truth haze-free images, low-pass filtered images, and high-pass filtered images. Based on this observation, we present a shared-encoder multi-decoder model ClarifyNet which employs interconnected parallelization. While training, ground-truth haze-free images, low-pass filtered images, and high-pass filtered images undergo multistage filter fusion and attention. By utilizing a weighted loss function composed of SSIM loss and L1 loss, we extract and propagate complementary features. We comprehensively evaluate ClarifyNet on I-HAZE, O-HAZE, Dense-Haze, NH-HAZE, SOTS-Indoor, SOTS-Outdoor, HSTS, and Middlebury datasets. We use PSNR and SSIM metrics and compare the results with previous works. For most datasets, ClarifyNet provides the highest scores. On using EfficientNet-B6 as the backbone, ClarifyNet has 18M parameters (model size of ∼71MB) and a throughput of 8 frames-per-second while processing images of size 2048x1024.

Objective: Automated cell nuclei segmentation is vital for the histopathological diagnosis of cancer. However, nuclei segmentation from “hematoxylin and eosin” (HE) stained “whole slide images” (WSIs) remains a challenge due to noise-induced intensity variations and uneven staining. The goal of this paper is to propose a novel deep learning model for accurately segmenting the nuclei in HE-stained WSIs.
Approach: We introduce FEEDNet, a novel encoder-decoder network that uses LSTM units and “feature enhancement blocks” (FE-blocks). Our proposed FE-block avoids the loss of location information incurred by pooling layers by concatenating the downsampled version of the original image to preserve pixel intensities. FEEDNet uses an LSTM unit to capture multi-channel representations compactly. Secondly, for datasets that provide class information, we train a multiclass segmentation model, which generates masks corresponding to each class at the output. Using
this information, we generate more accurate binary masks than that generated by conventional binary segmentation models.
Main results: We have thoroughly evaluated FEEDNet on CoNSeP, Kumar, and CPM-17 datasets. FEEDNet achieves the best value of PQ (panoptic quality) on CoNSeP and CPM-17 datasets and the second best value of
PQ on the Kumar dataset. The 32-bit floating-point version of FEEDNet has a model size of 64.90MB. With INT8 quantization, the model size reduces to only 16.51 MB, with a negligible loss in predictive performance on Kumar and CPM-17 datasets and a minor loss on the CoNSeP dataset.
Significance: Our proposed idea of generalized class-aware binary segmentation is shown to be accurate on a variety of datasets. FEEDNet has a smaller model size than the previous nuclei segmentation networks, which makes it suitable for execution on memory-constrained edge devices. The state-of-the-art predictive performance of FEEDNet makes it the most preferred network. The source code can be obtained from https://github.com/CandleLabAI/FEEDNet.

In this paper, we present novel bit-flip attack (BFA) algorithms for DNNs, along with techniques for defending against the attack. Our attack algorithms leverage information about the layer importance, such that a layer is considered important if it has high-ranked feature maps. We first present a classwisetargeted attack that degrades the accuracy of just one class in the dataset. Comparative evaluation with related works shows the effectiveness of our attack algorithm. We finally propose multiple novel defense strategies against untargeted BFAs. We comprehensively evaluate the robustness of both large-scale CNNs (VGG19, ResNext50, AlexNet and ResNet) and compact CNNs (MobileNet-v2, ShuffleNet, GoogleNet and SqueezeNet) towards BFAs. We also reveal a valuable insight that compact CNNs are highly vulnerable to not only well-crafted BFAs such as ours, but even random BFAs. Also, defense strategies are less effective on compact CNNs.

In recent years, there has been an enormous interest in using deep learning to classify underwater images to identify various objects like fishes, plankton, coral reefs, seagrass, submarines, and gestures of sea-divers. This classification is essential for measuring the water bodies' health and quality and protecting the endangered species. Further, it has applications in oceanography, marine economy and defense, environment protection, and underwater exploration and human-robot collaborative tasks. This paper presents a survey of deep learning techniques for performing the underwater image classification. We underscore the similarities and differences of several methods. We believe that underwater image classification is one of the killer application that would test the ultimate success of deep learning techniques. Towards realizing that goal, this survey seeks to inform researchers about state-of-the-art on deep learning on underwater images and also motivate them to push its frontiers forward. Index Terms-Deep neural networks, artificial intelligence, autonomous underwater vehicle, transfer learning.

Crowd counting is the process of counting or estimating the number of individuals in a crowd. There has been a rapid surge in the amount of Unmanned Aerial Vehicles (UAV) images over the last few years. However, efficient crowd counting techniques from UAV images have hardly come into the focus of the research community. Crowd counting from UAV images has its unique challenges compared to crowd counting from images in natural scenes. Moreover, solving the problem in real-time makes the task even harder. In this paper, we introduce an attention-based encoderdecoder model called Attention-based Real-time CrowdNet (ARCN). ARCN is a computationally efficient density estimationbased crowd counting model. It can perform crowd-counting from UAV images in real-time with high accuracy. Ours is the first work that proposes a real-time density map estimation and crowd counting model from drone-based images. The key idea of our work is to add "Convolution Block Attention Module" (CBAM) blocks in-between the bottleneck layers of the MobileCount architecture. The proposed ARCN model achieves an MAE of 19.9 and MSE of 27.7 on the DroneCrowd dataset. Also, on NVIDIA GTX 2080 Ti GPU, ARCN has a processing speed of 48 FPS, making it a real-time technique. The pretrained model is available at https://bit.ly/3na7LUy

As von Neumann computing architectures become increasingly constrained by data-movement overheads, researchers have started exploring in-memory computing (IMC) techniques to offset datamovement overheads. Due to the widespread use of SRAM, IMC techniques for SRAM hold the promise of accelerating a broad range of computing systems and applications. In this article, we present a survey of techniques for in-memory computing using SRAM memory. We review the use of SRAM-IMC for implementing Boolean, search and arithmetic operations, and accelerators for machine learning (especially neural networks) and automata computing. This paper aims to accelerate co-design efforts by informing researchers in both algorithm and hardware architecture fields about the recent developments in SRAM-based IMC techniques.

As ``deep neural networks'' (DNNs) achieve increasing accuracy, they are getting employed in increasingly diverse applications, including security-critical applications such as medical and defense. This immense use of DNNs has motivated the researchers to scrutinizingly study their security vulnerability and propose countermeasures, especially in the context of hardware security. In this paper, we present a survey of techniques for the hardware security of DNNs. For the research works, we highlight the threat-model, key idea for launching attack and defense strategies. We organize the works on salient categories to highlight their strengths and limitations. This paper aims to equip researchers with the knowledge of recent advances in DNN security and motivate them to think of security as the first principle.

"Unmanned aerial vehicles" (UAVs) are now being used for a wide range of surveillance applications. Specifically, the detection of on-ground vehicles from UAV images has attracted significant attention due to its potential in applications such as traffic management, parking lot management, and facilitating rescue operations in disaster zones and rugged terrains. This paper presents a survey of deep learning techniques for performing on-ground vehicle detection from aerial imagery captured using UAVs (also known as drones). We review the works in terms of their approach to improve accuracy and reduce computation overhead and their optimization objective. We show the similarities and differences of various techniques and also highlight the future challenges in this area. This survey will benefit researchers in the area of artificial intelligence, traffic surveillance, and applications of UAVs.

As convolutional neural networks (CNNs) improve in accuracy, their model size and computational overheads have also increased. These overheads make it challenging to deploy the CNNs on resource-constrained devices. Pruning is a promising technique to mitigate these overheads. In this paper, we propose a novel pruning technique called CURATING that looks at the pruning of CNNs as a multi-objective optimization problem. CURATING retains filters that i) are very different (less redundant) from each other in terms of their representation ii) have high saliency score i.e., they reduce the model accuracy drastically if pruned iii) are likely to produce higher activations. We treat a filter specific to an output channel as a probability distribution over spatial filters to measure the similarity between filters. The similarity matrix is leveraged to create filter embeddings, and we constrain our optimization problem to retain a diverse set of filters based on these filter embeddings. On a range of CNNs over well-known datasets, CURATING exercises a better or comparable tradeoff between model size, accuracy, and inference latency than existing techniques. For example, while pruning VGG16 on the ILSVRC-12 dataset, CURATING achieves higher accuracy and a smaller model size than the previous techniques.

In recent years, researchers have focused on reducing the model size and number of computations (measured as "multiply-accumulate" or MAC operations) of DNNs. The energy consumption of a DNN depends on both the number of MAC operations and the energy efficiency of each MAC operation. The former can be estimated at design time; however, the latter depends on the intricate data reuse patterns and underlying hardware architecture. Hence, estimating it at design time is challenging. This work shows that the conventional approach to estimate the data reuse, viz. arithmetic intensity, does not always correctly estimate the degree of data reuse in DNNs since it gives equal importance to all the data types. We propose a novel model, termed "data type aware weighted arithmetic intensity" (DI), which accounts for the unequal importance of different data types in DNNs. We evaluate our model on 25 state-of-the-art DNNs on two GPUs. We show that our model accurately models data-reuse for all possible data reuse patterns for different types of convolution and different types of layers. We show that our model is a better indicator of the energy efficiency of DNNs. We also show its generality using the central limit theorem.

The remarkable predictive performance of deep neural networks (DNNs) has led to their adoption in service domains of unprecedented scale and scope. However, the widespread adoption and growing commercialization of DNNs have underscored the importance of intellectual property (IP) protection. Devising techniques to ensure IP protection has become necessary due to the increasing trend of outsourcing the DNN computations on the untrusted accelerators in cloud-based services. The design methodologies and hyper-parameters of DNNs are crucial information, and leaking them may cause massive economic loss to the organization. Furthermore, the knowledge of DNN's architecture can increase the success probability of an adversarial attack where an adversary perturbs the inputs and alter the prediction. In this work, we devise a two-stage attack methodology "DeepPeep" which exploits the distinctive characteristics of design methodologies to reverse-engineer the architecture of building blocks in compact DNNs. We show the efficacy of "DeepPeep" on P100 and P4000 GPUs. Additionally, we propose intelligent design maneuvering strategies for thwarting IP theft through the DeepPeep attack and proposed "Secure MobileNet-V1". Interestingly, compared to vanilla MobileNet-V1, secure MobileNet-V1 provides a significant reduction in inference latency (≈60%) and improvement in predictive performance (≈2%) with very-low memory and computation overheads.

As the capabilities of mobile phones have increased, the potential of their negative use has also increased tremendously. For example, use of mobile phones while driving or in high-security zones can lead to accidents, information leaks and security breaches. In this paper, we use deep-learning algorithms viz., single shot multiBox detector (SSD) and faster-region based convolution neural network (Faster-RCNN), to detect mobile phone usage. We highlight the importance of mobile phone usage detection and the challenges involved in it. We have used a subset of State Farm Distracted Driver Detection dataset from Kaggle, which we term as Kag-gleDriver dataset. In addition, we have created a dataset on mobile phone usage, which we term as IITH-dataset on mobile phone usage (IITH-DMU). Although small, IITH-DMU is more generic than the KaggleDriver dataset, since it has images with higher amount of variation in foreground and background objects. Ours is possibly the first work to perform mobile-phone detection for a wide range of scenarios. On the KaggleDriver dataset, the AP at 0.5IoU is 98.97% with SSD and 98.84% with Faster-RCNN. On the IITH-DMU dataset, these numbers are 92.6% for SSD and 95.92% for Faster-RCNN. These pretrained models and the datasets are available at sites.google.com/view/mobile-phone-usage-detection.

Systolic architecture has widely been used for efficient processing of DNNs in both edge devices and datacenter servers. The number of processing elements (PEs) in a fixed-sized systolic accelerator is well matched for large and compute-bound DNNs; however, memory-bound DNNs suffer from PE underutilization and fail to achieve peak performance and energy efficiency.  To mitigate this underutilization, numerous dataflow techniques and micro-architectural heuristics have been proposed. In this work, we address this challenge at the algorithm front and propose data reuse aware co-optimization (DRACO), which improves the PE utilization of memory-bound DNNs without any additional need for dataflow/micro-architecture modifications. Furthermore, unlike the previous co-optimization methods, DRACO not only maximizes performance and energy efficiency but also boosts the representational power and improves the predictive performance of DNNs. We perform an extensive experimental study to understand the role of computational complexity and PE utilization on  (inference) latency optimization. The  key finding of this work  is that improving PE utilization does not always improve the performance of a DNN; it also depends on the computational overhead of improving PE utilization. 

To the best of our knowledge, DRACO is the first work that resolves the resource underutilization challenge at the algorithm level and demonstrates a trade-off between computational efficiency, PE utilization, and predictive performance of DNN. Compared to the state-of-the-art row stationary dataflow, DRACO achieves 41.8% and 42.6% improvement in average PE utilization and inference latency (respectively) with negligible loss in predictive performance in MobileNetV1 on a 64x64 systolic array. DRACO provides seminal insights for utilization-aware DNN design methodologies that can fully leverage the computation power of systolic array-based hardware accelerators.

The capability of the self-attention mechanism to model the long-range dependencies has catapulted its deployment in vision models. Unlike convolution operators, self-attention offers infinite receptive field and enables compute-efficient modeling of global dependencies. However, the existing state-of-the-art attention mechanisms incur high compute and/or parameter overheads, and hence unfit for compact convolutional neural networks (CNNs). In this work, we propose a simple yet effective "Ultra-Lightweight Subspace Attention Mechanism" (ULSAM), which infers different attention maps for each feature map subspace. We argue that leaning separate attention maps for each feature subspace enables multi-scale and multi-frequency feature representation, which is more desirable for fine-grained image classification. Our method of subspace attention is orthogonal and complementary to the existing state-of-the-arts attention mechanisms used in vision models. ULSAM is end-to-end trainable and can be deployed as a plug-and-play module in the pre-existing compact CNNs. Notably, our work is the first attempt that uses a subspace attention mechanism to increase the efficiency of compact CNNs. To show the efficacy of ULSAM, we perform experiments with MobileNet-V1 and MobileNet-V2 as backbone architectures on ImageNet-1K and three fine-grained image classification datasets. We achieve ≈13% and ≈25% reduction in both the FLOPs and parameter counts of MobileNet-V2 with a 0.27% and more than 1% improvement in top-1 accuracy on the ImageNet-1K and fine-grained image classification datasets (respectively). Code and trained models are available at https://github.com/Nandan91/ULSAM .

The number of groups (g) in group convolution (GConv) is selected to boost the predictive performance of deep neural networks (DNNs) in a compute and parameter efficient manner. However, we show that naive selection of g in GConv creates an imbalance between the computational complexity and degree of data reuse, which leads to suboptimal energy efficiency in DNNs. We devise an optimum group size model, which enables a balance between computational cost and data movement cost, thus, optimize the energy-efficiency of DNNs. Based on the insights from this model, we propose an "energy-efficient group convolution" (E2GC) module where, unlike the previous implementations of GConv, the group size (G) remains constant. Further, to demonstrate the efficacy of the E2GC module, we incorporate this module in the design of MobileNet-V1 and ResNeXt-50 and perform experiments on two GPUs, P100 and P4000. We show that, at comparable computational complexity , DNNs with constant group size (E2GC) are more energy-efficient than DNNs with a fixed number of groups (FgGC). For example, on P100 GPU, the energy-efficiency of MobileNet-V1 and ResNeXt-50 is increased by 10.8% and 4.73% (respectively) when E2GC modules substitute the FgGC modules in both the DNNs. Furthermore, through our extensive experimentation with ImageNet-1K and Food-101 image classification datasets, we show that the E2GC module enables a trade-off between generalization ability and representational power of DNN. Thus, the predictive performance of DNNs can be optimized by selecting an appropriate G. The code and trained models are available at https://github.com/iithcandle/E2GC-release.

The rise of deep-learning (DL) has been fuelled by the improvements in accelerators. Due to its unique features, the GPU continues to remain the most widely used accelerator for DL applications. In this paper, we present a survey of architecture and system-level techniques for optimizing DL applications on GPUs. We review techniques for both inference and training and for both single GPU and distributed system with multiple GPUs. We bring out the similarities and differences of different works and highlight their key attributes. This survey will be useful for both novice and experts in the field of machine learning, processor architecture and high-performance computing.

The rising overheads of data-movement and limitations of general-purpose processing architectures have led to a huge surge in the interest in ``processing-in-memory'' (PIM) approach and ``neural networks'' (NN) architectures. Spintronic memories facilitate efficient implementation of PIM approach and NN accelerators, and offer several advantages over conventional memories. In this paper, we present a survey of spintronic-architectures for PIM and NNs. We organize the works based on main attributes to underscore their similarities and differences. This paper will be useful for researchers in the area of artificial intelligence, hardware architecture, chip design and memory system.

In modern processors, data-movement consumes two orders of magnitude higher energy than a floating-point operation and hence, data-movement is becoming the primary bottleneck in scaling the performance of modern processors within the fixed power budget. Intelligent data-encoding techniques hold the promise of reducing the data-movement energy. In this paper, we present a survey of encoding techniques for reducing data-movement energy. By classifying the works on key metrics, we bring out their similarities and differences. This paper is expected to be useful for computer architects, processor designers and researchers in the area of interconnect and memory system design.

The recent trend in deep neural networks (DNNs) research is to make the networks more compact. The motivation behind designing compact DNNs is to improve energy efficiency since by virtue of having lower memory footprint, compact DNNs have lower number of off-chip accesses which improves energy efficiency. However, we show that making DNNs compact has indirect and subtle implications which are not well-understood. Reducing the number of parameters in DNNs increases the number of activations which, in turn, increases the memory footprint. We evaluate several recently-proposed compact DNNs on Tesla P100 GPU and show that their " activations to parameters ratio " ranges between 1.4 to 32.8. Further, the " memory-footprint to model size ratio " ranges between 15 to 443. This shows that a higher number of activations causes large memory footprint which increases on-chip/off-chip data movements. Furthermore, these parameter-reducing techniques reduce the arithmetic intensity which increases on-chip/off-chip memory bandwidth requirement. Due to these factors, the energy efficiency of compact DNNs may be significantly reduced which is against the original motivation for designing compact DNNs.

Driven by the trends of increasing core-count and bandwidth-wall problem, the size of last level caches (LLCs) has greatly increased. Since SRAM consumes high leakage power, researchers have explored use of non-volatile memories (NVMs) for designing caches as they provide high density and consume low leakage power. However, since NVMs have low write-endurance and the existing cache management policies are write variation-unaware, effective wear-leveling techniques are required for achieving reasonable cache lifetimes using NVMs. We present WriteSmoothing, a technique for mitigating intra-set write variation in NVM caches.  WriteSmoothing logically divides the cache-sets into multiple modules. For each module, WriteSmoothing collectively records number of writes in each way for any of the sets. It then periodically makes most frequently written ways in a module unavailable to shift the write-pressure to other ways in the sets of the module. Extensive simulation results have shown that on average, for single and dual-core system configurations, WriteSmoothing improves cache lifetime by 2.17X and 2.75X, respectively. Also, its implementation overhead is small and it works well for a  wide range of algorithm and system parameters.

Use of NVM (Non-volatile memory) devices such as ReRAM (resistive RAM) and STT-RAM (spin transfer torque RAM) for designing on-chip caches holds the promise of providing a high-density, low-leakage alternative to SRAM. However, low write endurance of NVMs, along with the write-variation introduced by existing cache management schemes may significantly limit the lifetime of NVM caches. We present LastingNVCache, a technique for improving lifetime of NVM caches by mitigating the intra-set write variation. LastingNVCache works on the key idea that by periodically flushing a frequently-written data-item, the next time the block can be made to load into a cold block in the set. Through this, the future writes to that data-item can be redirected from a hot block to a cold block, which leads to improvement in the cache lifetime. Microarchitectural simulations have shown that LastingNVCache provides 6.36X, 9.79X, and 10.94X  improvement in lifetime for single, dual and quad-core systems. Also, its implementation overhead is small and it outperforms a recently proposed technique for improving lifetime of NVM caches.