Sat.Sep 02, 2023 - Fri.Sep 08, 2023

article thumbnail

Threads: The inside story of Meta’s newest social app

Engineering at Meta

Earlier this year, a small team of engineers at Meta started working on an idea for a new app. It would have all the features people expect from a text-based conversations app, but with one very key, distinctive goal – being an app that would allow people to share their content across multiple platforms. We wanted to build a decentralized (or federated) app that would enable people to post content that is viewable by anyone on other social apps, and vice versa.

Email 143
article thumbnail

The 5 Stages of Your Data Analytics Journey

TDAN

Harnessing the power of data has become critical in today’s digital age when information is abundant and decision-making is critical in many aspects of business. Understanding your data may unearth hidden insights and move your business ahead, whether you’re a small startup or an established enterprise.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

How to Run Apache Kafka on Windows

Confluent

Kafka-on-Windows tutorials are everywhere, but most run Kafka directly on Windows.

116
116
article thumbnail

Is Your Data Ready for Generative AI?

Dataversity

Generative AI (GenAI) is all the rage in the world today, thanks to the advent of tools like ChatGPT and DALL-E. To their credit, these innovations are extraordinary. They’ve put the power of artificial intelligence and machine learning (AI/ML) into the hands of everyday users. However, these tools have also skewed our perceptions of what […] The post Is Your Data Ready for Generative AI?

article thumbnail

Arcadia: An end-to-end AI system performance simulator

Engineering at Meta

We’re introducing Arcadia, Meta’s unified system that simulates the compute, memory, and network performance of AI training clusters. Extracting maximum performance from an AI cluster and increasing overall efficiency warrants a multi-input system that accounts for various hardware and software parameters across compute, storage, and network collectively.

Topology 112
article thumbnail

Article: AI, ML, and Data Engineering InfoQ Trends Report - September 2023

InfoQ Articles

In this annual report, the InfoQ editors discuss the current state of AI, ML, and data engineering and what emerging trends you as a software engineer, architect, or data scientist should watch. We curate our discussions into a technology adoption curve with supporting commentary to help you understand how things are evolving.

article thumbnail

Establishing a new architecture practice in agile organisations

Ben Morris

How should you go about building an architecture practice in organisations where people are allowed to say no? Ultimately, this becomes an exercise in persuasion, pragmatism, and patience.

100
100

More Trending

article thumbnail

Using Chakra execution traces for benchmarking and network performance optimization

Engineering at Meta

Meta presents Chakra execution traces , an open graph-based representation of AI/ML workload execution, laying the foundation for benchmarking and network performance optimization. Chakra execution traces represent key operations, such as compute, memory, and communication, data and control dependencies, timing, and resource constraints. In collaboration with MLCommons , we are seeking industry-wide adoption for benchmarking.

Network 109
article thumbnail

Article: Building Kafka Event-Driven Applications with KafkaFlow

InfoQ Articles

KafkaFlow, a.NET open-source project, simplifies Kafka-based event-driven app development with features like middleware for message processing, enhancing maintainability, customization potential, and allowing developers to prioritize business logic.

article thumbnail

Design and Deployment Considerations for Deploying Apache Kafka on AWS

Confluent

Want to run Kafka on AWS? Our full tutorial provides expert recommendations on how to deploy, monitor, and manage Kafka clusters on AWS.

98
article thumbnail

Why Your Enterprise Needs Data-Centric Cybersecurity (and How to Achieve It)

Dataversity

Hardly a week goes by without news of another cybercrime. A recent example is the MOVEit breach, in which hackers stole data from customers of the MOVEit file transfer service. Between May and August, at least 600 organizations and 40 million users fell victim – with no apparent end in sight. One reason for the parade […] The post Why Your Enterprise Needs Data-Centric Cybersecurity (and How to Achieve It) appeared first on DATAVERSITY.

Education 111
article thumbnail

What’s New for Shared Clusters in Unity Catalog

databricks

We are thrilled to announce great enhancements to onboard more workloads to Unity Catalog clusters in shared access mode, Databricks' highly efficient, secure.

article thumbnail

What’s it like to write code at Meta?

Engineering at Meta

Ever wonder what it’s like to write code at Meta’s scale? On the latest episode of the Meta Tech Podcast , Meta engineer Pascal Hartig ( @passy ) sits down with Dustin Shahidehpour and Katherine Zak, two software engineers at Meta, about their careers and what it’s really like to ship code at Meta. Why does Meta have a monorepo?

article thumbnail

Easy and Instant Insurance Quotes Using Data Streaming with Confluent Cloud

Confluent

Using microservices, Confluent connectors, and stream processing on applicant data, historical data, actuarial tables, and predictive modeling to instantly generate insurance quotes.

article thumbnail

Strong AI/ML Must Be Founded on a Strong Data Strategy

Dataversity

The list of use cases powered by artificial intelligence (AI) and machine learning (ML) technologies is growing exponentially across nearly every business sector. Enterprises of all kinds are leveraging these advanced capabilities and scaling them through automation to improve business process management, sharpen organizational strategies, and reap more analytical and predictive insights from data for […] The post Strong AI/ML Must Be Founded on a Strong Data Strategy appeared first on DAT

article thumbnail

Announcing Databricks Bengaluru Development Center

databricks

In May this year, we opened our latest development center in Bengaluru, India. We've been busy building out our R&D teams in India.

93
article thumbnail

Heavy Networking 699: Connecting Multicloud Kubernetes Clusters With Virtual Application Networks

Packet Pushers

Virtual Application Networks, or VANs, are todays Heavy Networking topic. Our guest is Ted Ross, motive force behind the Skupper.io project. Skupper builds VANs in Kubernetes clusters that are conceptually like a VLAN or VPN, except that all the magic happens at layer 7. Skupper is based on the Advanced Message Queueing Protocol (AMQP), making it effectively a message bus used to interconnect application messages inside of mTLS tunnels running on top of whatever L3 network is available.

article thumbnail

Meet Bans, Our September 2023 Confluent Champion

Confluent

Meet Solutions Engineering Manager Bans Sagoo and learn what got him to Confluent—and how he’s driving better outcomes for customers.

article thumbnail

AI Will Make You 10X the Data Analyst, Not Take Your Job

Dataversity

AI isn’t here to replace data analysts; it’s here to unlock their full potential and dramatically scale their work. With AI as their powerful ally, data analysts can unlock latent value in data and lead the charge in the fourth industrial revolution. In a world where AI creates viral images and crafts hit music, it’s clear that we’re […] The post AI Will Make You 10X the Data Analyst, Not Take Your Job appeared first on DATAVERSITY.

article thumbnail

Retail Personalization with RFM Segmentation and the Composable CDP

databricks

Check out our Solution Accelerator for RFM Segmentation for more details and to download the notebooks. For retail brands, effective customer engagement depends.

92
article thumbnail

IPv6 Buzz 134: Revisiting Unique Local Addressing At The IETF

Packet Pushers

In today's IPv6 Buzz podcast, Ed, Scott, and Tom bring Nick Buraglio back on to the show to discussIPv6 Unique Local Addressingand the latest activity at the IETF to attempt to address both protocol and operational challenges associated withRFC 6724. In today's IPv6 Buzz podcast, Ed, Scott, and Tom bring Nick Buraglio back on to the show to discussIPv6 Unique Local Addressingand the latest activity at the IETF to attempt to address both protocol and operational challenges associated withRFC 6724

article thumbnail

Kmart on How to Turn Paper Receipts into Valuable Digital Data with Streaming

Confluent

At the Sydney tour stop of the Confluent Data in Motion Tour 2023, Kmart’s Principal Architect - Enterprise Technology, Duane Gomes, gave us the lowdown on how the company has used data streaming to power the digital loyalty program OnePass.

64
article thumbnail

The Three Things Data Practitioners Struggle With

Dataversity

Data Management has never been more critical than today. As AI grows more prominent, data initiatives are more important than ever. As the digital age propels us forward, the need for robust DataOps strategies becomes evident. These strategies, however, are not devoid of challenges. Data-focused practitioners have a unique relationship with data. They are grappling […] The post <strong>The Three Things Data Practitioners Struggle With</strong> appeared first on DATAVERSITY.

Cloud 64
article thumbnail

Introducing Databricks Bengaluru Development Center

databricks

In May this year, we opened our latest development center in Bengaluru, India. We've been busy building out our R&D teams in India.

73
article thumbnail

From Pre-trained to Fine-tuned: Nextdoor’s Path to Effective Embedding Applications

Nextdoor Engineering

Background The majority of ML models at Nextdoor are typically driven by a large number of features that are primarily either continuous or discrete in nature. The personalized features usually stem from historical aggregations or real-time summarization of interaction features, typically captured through logged tracking events. However, representing content through deep understanding using information behind it (text/image) is crucial for modeling nuanced user signals and better personalizing c

article thumbnail

Revolutionizing FinTech: Kredivo’s Growth with Data Streaming

Confluent

Explore how Confluent and Apache Kafka are reshaping the data streaming landscape in FinTech. Learn how these powerful tools are fostering efficiency and scalability, featuring customers like Kredivo Holdings.

59
article thumbnail

Heavy Wireless 010: Using Drones In Your WLAN Consulting

Packet Pushers

Drones can be a useful tool for outdoor wireless surveys. Drones can help an engineer figure out the best place to mount a radio, get line of sight between points, confirm antenna orientation, and save a lot of climbing up and down ladders. On today's Heavy Wireless, Keith Parsons speaks with Mike Wade, a wireless design engineer who uses drones in his survey work, about the certifications and requirements necessary to add a drone to your toolkit.

article thumbnail

Microsoft Dynamics Monitoring: CRM Performance Optimization

Obkio

Optimize Microsoft Dynamics performance with Microsoft Dynamics monitoring. Explore metrics, best practices & harness the power of Obkio network monitoring.

Network 52
article thumbnail

ARista 7280SR-48C6 routing table size

Network Engineering

i want to use a 7280SR-48C6 as edge switch and i read arista datasheet but i could not find how many ipv4 and ipv6 it can hold. can i have 4-5x IP transits with full tables and 30-40x private peering ? how many routes it can hold in fib and then install to rib?

52
article thumbnail

Data Driven Customer Retention Strategies

TDAN

Data driven marketing strategies supported by customer insights can significantly boost the customer experience, leading to higher customer retention rates.

article thumbnail

Day Two Cloud 209: Cloud Essentials – Virtual Private Clouds (VPCs)

Packet Pushers

Today's Day Two Cloud kicks off an occasional series on cloud essentials. For the first episode we discuss the Virtual Private Cloud (VPC). A VPC is an fundamental construct of a public cloud. It's essentially your slice of the shared cloud infrastructure, and you can launch and run other elements within a VPC to support your workload. Ned Bellavance walks through key VPC components including regions and AZs, networking and IP addressing, paid add-ons, data egress and associated charges, monitor

Cloud 52
article thumbnail

A Contrarian View of Software Architecture

UML Zone

The presenter has spent an inordinate amount of time the past half decade across multiple companies working with very large, long-running enterprise software systems. Especially in long-running, constantly changing systems, you want the code and the software architecture to be easy to understand, relatively painless to extend or modify, and when advantageous, be simple to […] The post A Contrarian View of Software Architecture first appeared on Software Architecture & Design.

52
article thumbnail

Single Vendor SASE vs. the Alternatives: Navigating Your Options

CATO Networks

SASE sets the design guidelines for the convergence of networking and security as a cloud service. With SASE, enterprises can achieve operational simplicity, reliability, and adaptability. Unsurprisingly, since Gartner defined SASE in 2019, vendors have been repositioning their product offerings as SASE. So, what are the differences between the recommended single-vendor SASE approach and other SASE alternatives?

SASE 52
article thumbnail

The Data-Centric Revolution: “RDF is Too Hard”

TDAN

We hear this a lot. We hear it from very smart people. Just the other day we heard someone say they had tried RDF twice at previous companies and it failed both times. (RDF stands for Resource Description Framework,[1] which is an open standard underlying many graph databases).

article thumbnail

Network Break 445: Juniper Pairs With ChatGPT, Microsoft To Unpair Teams In The EU

Packet Pushers

Today on Network Break we discuss Juniper integrating ChatGPT with its AI digital assistant, Microsoft's plan to unbundle Teams in the EU to fend off regulators, financial results from soon-to-be-paired Broadcom and VMware, a 5G follow-up, and more. Today on Network Break we discuss Juniper integrating ChatGPT with its AI digital assistant, Microsoft's plan to unbundle Teams in the EU to fend off regulators, financial results from soon-to-be-paired Broadcom and VMware, a 5G follow-up, and more.

5G 52
article thumbnail

Shared Clusters in Unity Catalog for the win: Introducing Cluster Libraries, Python UDFs, Scala, Machine Learning and more

databricks

We are thrilled to announce that you can run even more workloads on Databricks’ highly efficient multi-user clusters thanks to new security and g.

article thumbnail

HighCPU-Power RPS Process

Network Engineering

By chance no one encountered such a problem, periodically due to the "Power RPS Process" process, the processor is very heavily loaded,accordingly, I constantly receive alerts about a high delay to the switch, I dont notice any problems with user traffic. Switch: WS-C2960X-24PS-L 15.2(2)E5 C2960X-UNIVERSALK9-M PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 181 1367570308 34977206 39099 44.46% 25.47% 20.32% 0 Power RPS Proces Unfortunately, I couldn't find anything similar on