Abstract: The present disclosure provides systems and methods for estimating audience member engagement with content, or distinguishing between users consuming content and users who have become distracted or have left. User presence or attention may be estimated from user interactions with or skipping of content, with the user interactions either compared to high quality engagement data from small audience measurement panels, or extrapolated based on a temporal-engagement curve. An attention gap may be estimated, representing users that were not present for or not engaged with or paying attention to a presentation of content at a client device. This allows the measurement system to distinguish between users who consumed and potentially enjoyed the content, and users who did not, even as client devices of both sets of users receive and present the content items.

Abstract: Aspects of the present disclosure relate to synchronizing route updates. In one aspect, one or more computing devices, such as a router or a centralized controller, may receive a notification of a planned topology change to the network. The topology change may affect a link between nodes. Further, the one or more computing devices may determine one or more paths associated with the link and determine one or more user nodes associated with each path. The one or more computing devices may send an instruction message associated with the planned topology change to each user node of each path and receive an acknowledgment message in response to the instruction message. Based on the information in the acknowledgement message, the one or more computing devices may determine whether to effect the planned topology change.

Abstract: A method includes, but is not limited to any combination of: determining a plurality of third-party content elements based in part on information associated with a request for third-party content. The request for third-party content may be received from a web browser displaying a first webpage to a user. A first element from the plurality of determined third-party content elements is transmitted to the web browser. Upon receiving from the web browser a second request for third-party content including user interaction data with the first element, data associated with the determined plurality of third-party content elements is updated based at least in part on the user interaction data. A second third-party element from the plurality of updated third-party content elements is transmitted to the web browser.

Abstract: Selection and serving of content items may include receiving data indicative of a status of an energy source of a device with a request for a content item. A first received content item may be associated with a first energy consumption level and a second received content item may be associated with a second energy consumption level. The accessed content items are responsive to the request for a content item. The first energy consumption level may be higher than the second energy consumption level. The first content item or the second content item may be selected based, at least in part, on the received data indicative of the status of the energy source of the device, and data to display the selected content item may be provided to the device.

Abstract: Methods, systems, and apparatus for controlling smart devices are described. In one aspect a method includes receiving image data for an image captured by a camera of a mobile device of a user and determining that the image depicts at least one of a smart device or a physical control for the smart device. In response to determining that that the image depicts a smart device or a physical control for the smart device, identifying one or more user interface controls for controlling the smart device, and generating and presenting, at a display of the mobile device, the one or more user interface controls for controlling the smart device. The method can further include detecting, at the display of the mobile device, user interaction with at least one of the one or more user interface controls, and controlling the smart device based on the detected user interaction.

Abstract: This relates to apparatus, systems, methods, and related computer program products for providing home security/smart-home objectives. More particularly, this relates to intelligent, multi-sensing, network-connected devices that communicate with each other and/or with a central server or a cloud-computing system to provide any of a variety of useful home security/smart-home objectives, including managing ingress and egress through any smart door of a smart-home.

Abstract: Methods, systems, and media for providing media guidance with contextual controls are provided. In some embodiments, the method comprises: causing a user interface to be presented that includes media interface elements, wherein each of the media interface elements is a selectable object that represents at least one media content item; determining that a media interface element from the media interface elements has been selected; in response to determining that the media interface element has been selected, causing the at least one media content item corresponding to the selected media interface element to be presented in a portion of the user interface; and concurrently with causing the at least one media content item to be presented, causing a control interface to be presented that is associated with the selected media interface element.

Abstract: Systems, devices, and methods of wirelessly communicatively coupling a wearable computing device to a host computing device are disclosed. The wearable computing device transmits a first advertisement packet via a first personal area network and a second advertisement packet via a second personal area network. The wearable computing device receives a response to one of the first advertisement packet and the second advertisement packet. Based on the response, the wearable computing device selects one of a first coupling procedure and a second coupling procedure to perform. The wearable computing device performs the selected coupling procedure with the host computing device.

Abstract: Systems and methods of passing data via resource identifiers for resources are provided herein. A data processing system can receive a first request from a client device responsive to an interaction with a content item presented on a first resource. The content item can include a resource identifier referencing a second resource accessible via an application to be installed. The first request can include the resource identifier and a device identifier. The data processing system can receive a second request from the client device responsive to an execution of the installed application. The second request can include the device identifier and an application identifier. The data processing system can determine that the application is to receive the request identifier. The data processing system can provide the resource identifier to direct the application to access the second resource for presentation via the application.

Abstract: Methods and apparatus related to determining points of interest based on content of one or more sources associated with a user. Relevance scores are determined for the points of interest based on additional information associated with the user and the content. Point of interest information that is related to the points of interest is provided based on the determined relevance scores.

Abstract: Systems and methods for providing interaction data in response to a query from a content provider are provided. One method includes receiving a query from the content provide including a plurality of parameters relating to interactions with a content item of the content provider. The method includes retrieving a set of log file records using the included plurality of parameters and generating a current set of records identifying interaction data with the content item. The method includes comparing the current set of records to a previous set of records generated by the one or more processors for a previous query from the content provider and determining a similarity between the current set of records and the previous set of records. The method includes modifying, if the similarity is above a threshold, the current set of records and providing the modified set of records to the content provider.

Abstract: Immersive web-based simulator for digital assistant-based applications is provided. A system can provide, for display in a web browser, an inner iframe configured to load, in a secure, access restricted computing environment, an application configured to integrate with a digital assistant. The application can be provided by a third-party developer device. The system can provide, for display in a web browser, an outer iframe configured with a two-way communication protocol to communicate with the inner iframe. The system can provide a state machine to identify a current state of the application loaded in the inner frame, and load a next state of the application responsive to a control input.

Abstract: Interface and mode selection for digital action execution is provided. For example, a system loads a script library embedded in an electronic resource. The system determines a historic level of engagement between a client computing device and one or more digital assistants. The system selects, based on a first property of the client computing device and the historic level of engagement, a type of digital interface. The system generates, based on the type of digital interface, a digital interface with the call-to-action. The system determines, responsive to an instruction to execute the call-to-action, a mode of execution. The system selects a digital assistant and a second client device to execute the call-to-action. The system transmits the call-to-action to the second client device for execution.

Abstract: Implementations relate to interactions, over a wireless communication modality, between an automated assistant of an automobile computer system, and separate client device(s). In some of those implementations, the client device(s) can be paired with the automobile computer system, and the client device(s) and the automobile computer system can include separate instances of an automated assistant, which can be associated with different user accounts. In some additional or alternative implementations, particular actions to be performed by a local application of a client device can be advanced via user interaction with the automobile automated assistant, despite a counterpart to the local application not being installed directly on automobile computer system.

Abstract: The various implementations described herein include methods, devices, and systems for attending to a presenting user. In one aspect, a method is performed at an electronic device that includes an image sensor, microphones, a display, processor(s), and memory. The device (1) obtains audio signals by concurrently receiving audio data at each microphone; (2) determines based on the obtained audio signals that a person is speaking in a vicinity of the device; (3) obtains video data from the image sensor; (4) determines via the video data that the person is not within a field of view of the image sensor; (5) reorients the electronic device based on differences in the received audio data; (6) after reorienting the electronic device, obtains second video data from the image sensor and determines that the person is within the field of view; and (7) attends to the person by directing the display toward the person.

Abstract: Systems and methods are described to suggest new content to a writer. An intelligent tool is initiated to obtain content of a document that is being written by an author. The intelligent tool then determines a topic of the content of the document, and identifies one or more predictive topics related to the determined topic. The one or more predictive topics are different from the topic. The intelligent tool then performs a content search based on the topic and the one or more predictive topics, and provides results from the content search to the author.

Abstract: A system and method is provided for displaying surfaces of an object from a vantage point different from the vantage point from which imagery of the object was captured. In some aspects, imagery may be generated for display by combining visual characteristics from multiple source images and applying greater weight to the visual characteristics of some of the source images relative to the other source images. The weight may be based on the orientation of the surface relative to the location from which the image was captured and the location from which the object will be displayed.

Abstract: Systems and methods are provided for a personalized entity repository. For example, a computing device comprises a personalized entity repository having fixed sets of entities from an entity repository stored at a server, a processor, and memory storing instructions that cause the computing device to identify fixed sets of entities that are relevant to a user based on context associated with the computing device, rank the fixed sets by relevancy, and update the personalized entity repository using selected sets determined based on the rank and on set usage parameters applicable to the user. In another example, a method includes generating fixed sets of entities from an entity repository, including location-based sets and topic-based sets, and providing a subset of the fixed sets to a client, the client requesting the subset based on the client's location and on items identified in content generated for display on the client.

Abstract: In some implementations, a method includes, for each of multiple hosts: identifying visual leaf pages hosted by the host that are each a web page including image data defining an image or a video that is prominently displayed relative to all other content of the web page, identifying a set of hub pages hosted by the host that each link to at least one of the visual leaf pages through an image-based link, and for each hub page, generating cluster data representing the visual leaf pages to which the hub page links by determining, for each visual leaf page, a set of feature values that each indicate pre-defined features of the visual leaf page, and generating, from the sets of feature values, a set of central feature values as the cluster data for the hub page that indicate a central tendency of each respective pre-defined feature.

Abstract: Techniques are described herein for assembling/evaluating automated assistant responses for privacy concerns. In various implementations, a free-form natural language input may be received from a first user and may include a request for information pertaining to a second user. Multiple data sources may be identified that are accessible by an automated assistant to retrieve data associated with the second user. The multiple data sources may collectively include sufficient data to formulate a natural language response to the request. Respective privacy scores associated with the multiple data sources may be used to determine an aggregate privacy score associated with responding to the request. The natural language response may then be output at a client device operated by the first user in response to a determination that the aggregate privacy score associated with the natural language response satisfies a privacy criterion established for the second user with respect to the first user.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for allocating computing resources. In one aspect, a method includes receiving intent data specifying one or more computing services to be hosted by a computing network, requested characteristics of computing resources for use in hosting the computing service, and a priority value for each requested characteristic. A budget constraint is identified for each computing service. Available resources data is identified that specifies a set of available computing resources. A resource allocation problem for allocating computing resources for the one or more computing resources is generated based on the intent data, each budget constraint, and the available resources data. At least a portion of the set of computing resources is allocated for the one or more computing services based on results of evaluating the resource allocation problem to meet a particular resource allocation objective.

Abstract: A method to incorporate a first edit and a second edit into an electronic document includes receiving the first edit and the second edit having a shared position in the electronic document, and determining whether the second edit is privileged relative to the first edit. The method further includes responsive to determining that the second edit is not privileged relative to the first edit, requiring the second edit to be independently accepted prior to incorporating the second edit into the electronic document, and responsive to determining that the second edit is privileged relative to the first edit, incorporating the second edit into the electronic document without requiring the second edit to be independently accepted, and responsive to receiving an acceptance of the first edit, causing the first edit and the second edit to be incorporated into the electronic document.

Abstract: Methods, systems, and apparatus for operating a system of qubits. In one aspect, a method includes operating a first qubit from a first plurality of qubits at a first qubit frequency from a first qubit frequency region, and operating a second qubit from the first plurality of qubits at a second qubit frequency from a second first qubit frequency region, the second qubit frequency and the second first qubit frequency region being different to the first qubit frequency and the first qubit frequency region, respectively, wherein the second qubit is diagonal to the first qubit in a two-dimensional grid of qubits.

Abstract: Systems and methods for enhancing search results for improved readability are provided. A search result is identified using a set of query terms. The search result may be first-party content or third-party content. The search result includes a first textual portion and a second textual portion. Each of the set of query terms that appear in the first textual portion are highlighted in the first textual portion. Each of the set of query terms that appear in the second textual portion but not in the first textual portion are highlighted in the second textual portion. A query response including the highlighted search result is generated and provided to a client device.

Abstract: Systems and methods are shown for providing private local sponsored content selection and improving intelligence models through distribution among mobile devices. This allows greater data gathering capabilities through the use of the sensors of the mobile devices as well as data stored on data storage components of the mobile devices to create predicted models while offering better opportunities to preserve privacy. Locally stored profiles comprising machine intelligence models may also be used to determine the relevance of the data gathered and in improving an aggregated model for identifying the relevance of data and the selection of sponsored content items. Distributed optimization is used in conjunction with privacy techniques to create the improved machine intelligence models. Publishers may also benefit from the improved privacy by protecting the statistics of type or volume of sponsored content items shown with publisher content.

Abstract: Methods and systems are provided for determining the intent of a recommendation made by a user of a mobile application where the application includes a plurality of separable components, any one or more of which the recommendation can apply to. An application in which a user recommendation control is provided for presentation to a user also includes a tag indicating how a recommendation of the application should be interpreted with respect to the components included therein. The tag can be set by the application developer and can be in the form of text (e.g., a keyword or term) or a uniform resource locator (URL). Where a tag references multiple components of an application, a recommending user can be presented with a recommendation intent query. The recommendation intent query allows a user to designate one or more components of the application to which the user's recommendation should be attributed.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for modifying a computational graph to include send and receive nodes. Communication between unique devices performing operations of different subgraphs of the computational graph can be handled efficiently by inserting send and receive nodes into each subgraph. When executed, the operations that these send and receive nodes represent may enable pairs of unique devices to conduct communication with each other in a self-sufficient manner. This shifts the burden of coordinating communication away from the backend, which affords the system that processes this computational graph representation the opportunity to perform one or more other processes while devices are executing subgraphs.

Abstract: Secure digital assistant integration with web pages is provided. The system receives an intent manifest data structure that maps actions of a digital assistant with link templates of an electronic resource developed by a third-party developer device. The system validates the electronic resource based on the intent manifest data structure. The system receives, from a data exchange component of an iframe of the electronic resource loaded by a client computing device, an identifier of the client computing device. The system receives a foreground state of the electronic resource from an onsite state sharing API. The system selects a data value for a parameter based on the foreground state and the intent manifest data structure. The system provides the data value. An authorization component generates an authorization prompt, receives input, and transmits the data value to an onsite intent execution API of the electronic resource to execute an action.

Abstract: In one example, a method includes method comprising: receiving audio data generated by a microphone of a current computing device; identifying, based on the audio data, one or more computing devices that each emitted a respective audio signal in response to speech reception being activated at the current computing device; and selecting either the current computing device or a particular computing device from the identified one or more computing devices to satisfy a spoken utterance determined based on the audio data.

Abstract: Systems and methods of dynamically varying the intensity of providing content items in a remarketing campaign based on tracking client device interactions are provided. The system can assign an account identifier to a first segment for a pre-conversion model, responsive to receiving a first interaction associated with a content provider from a client device. The system can assign the account identifier to a second segment for the pre-conversion model, responsive to receiving a second interaction. The system can assign the account identifier to a third segment, responsive to receiving a third interaction. The third interaction can include a conversion event. The system can generate a post-conversion model based on the third segment and the pre-conversion model. The system can determine an intent index for the account identifier based on the post-conversion model. The system can store the account identifier into an interest cluster based on the intent index.

Abstract: Systems and methods for enabling voice-based interactions with electronic devices can include a data processing system maintaining a plurality of device action data sets and a respective identifier for each device action data set. The data processing system can receive, from an electronic device, an audio signal representing a voice query and an identifier. The data processing system can identify, using the identifier, a device action data set. The data processing system can identify a device action from device action data set based on content of the audio signal. The data processing system can then identify, from the device action dataset, a command associated with the device action and send the command to the for execution device for execution.

Abstract: Systems and methods for transforming grayscale images into color images using deep neural networks are described. One of the systems include one or more computers and one or more storage devices storing instructions that, when executed by one or more computers, cause the one or more computers to implement a coloring neural network, a refinement neural network, and a subsystem. The coloring neural network is configured to receive a first grayscale image having a first resolution and to process the first grayscale image to generate a first color image having a second resolution lower than the first resolution. The subsystem processes the first color image to generate a set of intermediate image outputs. The refinement neural network is configured to receive the set intermediate image outputs, and to process the set of intermediate image outputs to generate a second color image having a third resolution higher than the second resolution.

Abstract: The systems and methods of the present disclosure generally relate to a data processing system that can identify and surface alternative requests when presented with ambiguous, unclear, or other requests to which a data processing system may not be able to respond. The data processing system can improve the efficiency of network transmissions to reduce network bandwidth usage and processor utilization by selecting alternative requests that are responsive to the intent of the original request.

Abstract: In some implementations, an utterance is determined to include a particular user speaking a hotword based at least on a first set of samples of the particular user speaking the hotword. In response to determining that an utterance includes a particular user speaking a hotword based at least on a first set of samples of the particular user speaking the hotword, at least a portion of the utterance is stored as a new sample. A second set of samples of the particular user speaking the utterance is obtained, where the second set of samples includes the new sample and less than all the samples in the first set of samples. A second utterance is determined to include the particular user speaking the hotword based at least on the second set of samples of the user speaking the hotword.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for distributing content items. In one aspect, a method includes presenting an immersive content item at a client device. The immersive content item includes a first presentation interface that includes a scrollable image portion that presents an image having a first interaction point and a preview graphic indicating the availability of other images for presentation. The immersive content item also includes a second presentation interface that is initiated upon interaction with the first interaction point. The second presentation interface includes a scrollable image portion that presents an image from a plurality of images and an image sequencing indicator.

Abstract: A user registers for an account with an account management system, configures account settings to permit the account management system to receive user computing device data from a user computing device associated with the user, and logs into the account via the user computing device. The account management system receives a user voice purchase command and determines a purchase command context based on the received user computing device data. The account management system identifies a product that the user desires to purchase based on the purchase command context and directs the user computing device web browser to a merchant website to set up a transaction for the identified product.

Abstract: A location of an object of interest (205) is determined using both observations and non-observations. Numerous images (341-345) are stored in a database in association with image capture information, including an image capture location (221-225). Image recognition is used to determine which of the images include the object of interest (205) and which of the images do not include the object of interest. For each of multiple candidate locations (455) within an area of the captured images, a likelihood value of the object of interest existing at the candidate location is calculated using the image capture information for images determined to include the object of interest and using the image capture information for images determined not to include the object of interest. The location of the object is determined using the likelihood values for the multiple candidate locations.

Abstract: A method for determining an architecture for a task neural network configured to perform a particular machine learning task is described.

Abstract: Systems and methods of selecting content based on image data are provided. A system can receive an image captured by a camera of the computing device. The system can analyze the image to identify a pattern that matches a predetermined pattern of an object stored in an image pattern database comprising a plurality of predetermined patterns of objects. The system can identify one or more image keywords from the image based on the predetermined pattern of the object that matches the image. The system can select, based on a comparison of the one or more image keywords with one or more keywords of each of a plurality of content items, a content item. The system can provide, to the computing device, the content item to cause the computing device to present the content item.

Abstract: A system of reducing transmissions of packetized data in a voice activated data packet based computer network environment is provided. A natural language processor component can parse an input audio signal to identify a request and a trigger keyword. Based on the input audio signal, a direct action application programming interface can generate a first action data structure, and a content selector component can select a content item. An interface management component can identify candidate interfaces and determine if prior instances of the packetized data was transmitted to the candidate interfaces. The interface management component can prevent the transmission of the packetized data if determined to be redundant, such as having previously received the data, and instead transmit it to a separate client device of a different device type.

Abstract: A voice to text model used by a voice-enabled electronic device is dynamically and in a context-sensitive manner updated to facilitate recognition of entities that potentially may be spoken by a user in a voice input directed to the voice-enabled electronic device. The dynamic update to the voice to text model may be performed, for example, based upon processing of a first portion of a voice input, e.g., based upon detection of a particular type of voice action, and may be targeted to facilitate the recognition of entities that may occur in a later portion of the same voice input, e.g., entities that are particularly relevant to one or more parameters associated with a detected type of voice action.

Abstract: Sensor data having values received from several sensors of a mobile device and response data associated with the sensor data may be used in the determination or training of a predictive model. Received sensor data may be input into the predictive model, and the output of the predictive model may be used in the selection and serving of content items to the mobile device. Data to effect presentation of the selected content item may be outputted to the mobile device to effect presentation. In some instances, the predictive model may be updated using the received plurality of values. The updated predictive model may be used in the selection of a subsequent content item for the mobile device. In other implementations, historical sensor data may be used with the set of received sensor data as input for the predictive model.

Abstract: Systems and methods of selecting digital platforms for execution of voice-based commands are provided. The system receives an application that performs an action associated with a service via digital platforms. The system debugs the application to validate parameters of the action on at least two platforms of the digital platforms. The system receives data packets comprising an input audio signal detected by a sensor of a client device, and parses the input audio signal to identify the action and the service. The system selects a first platform from the digital platforms to perform the action. The system initiates, responsive to selection of the first platform, an interactive data exchange to populate parameters of an action data structure corresponding to the action. The system executes the action via the selected platform using the action data structure.

Abstract: A system of multi-modal transmission of packetized data in a voice activated data packet based computer network environment is provided. A natural language processor component can parse an input audio signal to identify a request and a trigger keyword. Based on the input audio signal, a direct action application programming interface can generate a first action data structure, and a content selector component can select a content item. An interface management component can identify first and second candidate interfaces, and respective resource utilization values. The interface management component can select, based on the resource utilization values, the first candidate interface to present the content item. The interface management component can provide the first action data structure to the client computing device for rendering as audio output, and can transmit the content item converted for a first modality to deliver the content item for rendering from the selected interface.

Abstract: Methods, systems, and apparatus include computer programs encoded on a computer-readable storage medium, including a method for providing content. A user of an initiating device is identified. Profile information for the identified user is located. The initiating device includes a display for presenting content to the user. An indication is received from an application running on the initiating device of an intent by the user to receive a first content item on a separate but co-located presentation device having enhanced presentation characteristics for presenting content. Additional content items are selected for delivery along with the first content item. The selection includes identifying a second different content item based on the profile information for the identified user and the enhanced presentation characteristics.

Abstract: The various implementations described herein include a video camera assembly that includes: (1) a housing; (2) an image sensor positioned within the housing and having a field of view corresponding to a scene in the smart home environment; and (3) a concave-shaped front face positioned in front of the image sensor such that light from the scene passes through the front face prior to entering the image sensor; where the front face includes: (a) an inner section corresponding to the image sensor; and (b) an outer section between the housing and the inner section, the outer section having a concave shape that extends from an outer periphery of the outer section to an inner periphery of the outer section; and where the concave shape extends around an entirety of the outer periphery.

Abstract: An HMD device identifies a pose of the device and identifies a subset of a plurality of camera viewpoints of a light-field based on the pose. The HMD device interpolates image data of the light-field based on the pose and the subset of the plurality of camera viewpoints to generate an interpolated view; and displays at the HMD device an image based on the interpolated view. By interpolating based on the subset of camera viewpoints, the HMD device can reduce processing overhead and improve the user experience.

Abstract: This disclosure relates to systems and methods for displaying of thumbnails of content to be played after viewing of a video advertisement. A method includes generating a thumbnail associated with video content, the video content to be provided for playback after playback of a video advertisement, and presenting the thumbnail during the playback of the video advertisement to inform a user of the video content, wherein a skip counter indicative of time left until skipping of the video advertisement is enabled is provided, and wherein the time left is less than a remaining playing time of the video advertisement.